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United  States  Senate 
Washington,  D.C.  20501 

Dear  Mr.  Chairman: 

I  am  pleased  to  present  to  the  Committee  on  Labor  and  Public  Welfare 
the  final  report  of  a  study  conducted  by  the  Institute  of  Medicine,  National 
Academy  of  Sciences,  pursuant  to  Section  4  of  the  Health  Maintenance 
Organization  Act  of  1973  (P.L.  93-222).     That  provision  requested  the  Secretary 
of  Health,  Education,  and  Welfare  to  arrange  for  a  comprehensive  study  of 
health  care  quality  assurance  programs. 

The  Institute  of  Medicine  was  asked  to  conduct  a  limited  version  of  that 
study  which  had  the  following  objectives:     the  description  and  assessment 
of  the  effect  of  operational  quality  review  programs,  based  on  existing  infor- 
mation and  supplemented  by  observations  and  data  obtained  in  selected  site 
visits;  and  a  more  detailed  examination  of  several  topics,  designated 
"priority  areas"  because  of  their  importance  in  determining  the  effectiveness 
of  quality  assurance  programs.     These  areas  include  outcome-oriented  approaches 
to  quality  assurance,  quality  assurance  for  ambulatory  care,  quality  assurance 
for  long-term  care,  methods  for  changing  behavior  patterns  of  health  care 
providers,  and  patient  and  consumer  involvement  in  quality  assurance  programs. 
The  findings  and  recommendations  for  both  objectives  are  presented  in  this 
volume . 

An  additional  study  objective,  requested  by  the  Department  of  Health, 
Education,  and  Welfare,  required  an  assessment  of  the  reliability  of  hospital 
utilization  information  generated  by  private  abstracting  services  and  based 
on  abstracts  of  the  hospital  medical  record.     The  purpose  was  to  determine 
the  usefulness  of  such  information  for  evaluating  the  impact  of  Professional 
Standards  Review  Organizations.     That  study  will  be  reported  in  a  separate 
vo 1 ume . 

We  shall  be  pleased  to  discuss  this  report  in  greater  detail  with  the 
members  and  staff  of  your  committee. 


Sincerely  yours, 


David  A.  Hamburg,  M.D. 
President 
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Dear  Mr.  Chairman: 
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Commerce  the  final  report  of  a  study  conducted  by  the  Institute  of  Medicine, 
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visits;  and  a  more  detailed  examination  of  several  topics,  designated 
"priority  areas"  because  of  their  importance  in  determining  the  effectiveness 
of  quality  assurance  programs.     These  areas  include  outcome-oriented  approaches 
to  quality  assurance,  quality  assurance  for  ambulatory  care,  quality  assurance 
for  long-term  care,  methods  for  changing  behavior  patterns  of  health  care 
providers,  and  patient  and  consumer  involvement  in  quality  assurance  programs. 
The  findings  and  recommendations  for  both  objectives  are  presented  in  this 
volume . 

An  additional  study  objective,  requested  by  the  Department  of  Health, 
Education,  and  Welfare,  required  an  assessment  of  the  reliability  of  hospital 
utilization  information  generated  by  private  abstracting  services  and  based 
on  abstracts  of  the  hospital  medical  record.     The  purpose  was  to  determine 
the  usefulness  of  such  information  for  evaluating  the  impact  of  Professional 
Standards  Review  Organizations.     That  study  will  be  reported  in  a  separate 
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Dear  Dr.  Cooper: 

I  am  pleased  to  present  to  the  Department  of  Health,  Education,  and 
Welfare  the   final  report  of  a  study  conducted  by  the  Institute  of  Medicine, 
National  Academy  of  Sciences,  pursuant  to  Section  4  of  the  Health  Maintenance 
Organization  Act  of  1973  (P.L.  93-222).     That  provision  requested  the  Secretar 
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FOREWORD 


Assuring  the  quality  of  medical  care  is  a  major  innovation  in  the  delivery 
of  health  services  in  the  United  States.     Indeed,  no  other  nation  has  embarked 
upon  such  a  large  scale  effort  to  review  the  quality  of  health  services. 
Professional  Standards  Review  Organizations  are  the  vehicles  through  which 
this  country  is  currently  attempting  to  monitor  the  quality  of  services  pro- 
vided to  beneficiaries  of  the  Medicare,  Medicaid,  and  Maternal  and  Child 
Health  programs.     It  is  much  too  early  to  make  a  definitive  evaluation  of  the 
success  of  this  massive  effort.     Nevertheless,  the  United  States  Congress, 
the  Institute  of  Medicine,  and  other  groups  throughout  the  country  are  deeply 
involved  in  these  evolving  mechanisms  for  assuring  quality  of  care,  as  well 
as  alternative  methods  and  approaches. 

The  complexities  of  quality  assurance  were  noted  in  the  Institute  of  Medicine's 
1974  policy  statement,  "Advancing  the  Quality  of  Health  Care,"  which  outlined 
the  state-of-the-art  at  that  time.     When  Congress  requested  a  study  of  health 
care  quality  assurance  programs  as  part  of  the  Health  Maintenance  Organization 
Act  of  1973,   it  was  natural  that  those  charged  with  implementing  the 
Congressional  mandate  should  turn  to  the  Institute  of  Medicine  for  assistance 
in  conducting  a  limited  version  of  the  requested  study.     The  report  that 
follows  assesses  the  current  state  of  quality  assurance  by  reviewing  ongoing 
programs  and  existing  literature.     Recommendations  are  made  for  future  direc- 
tions in  quality  assurance. 

As  is  customary  for  Institute  of  Medicine  reports,   a  steering  committee  was 
appointed,  consisting  of  both  Institute  members  and  non-members  expert  in 
the  field.     The  steering  committee  guided  and  approved  the  conduct  of  the 
study  and  the  final  report,  which  was  reviewed  by  the  Council  of  the  Institute 
of  Medicine  and  the  National  Academy  of  Sciences.     It  was  my  privilege  to  work 
with  a  dedicated  and  competent  staff  and  a  stimulating  steering  committee 
throughout  the  study. 

The  report  is  not  the  final  statement  on  quality  assurance  programs,   for  they 
will  continue  to  change  rapidly.     Nevertheless,  we  trust  that  it  will  further 
the  development  of  this  major  national  effort  to  improve  health  and  health 
care . 


Robert  J.  Haggerty,  M.D. 
Chairman,   Steering  Committee 
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visit  them  and  learn  from  their  experiences,   this  study  would  not  have  been 
possible.     Both  the  study  staff  and  steering  committee  wish  to  thank  them 
for  generously  donating  their  time,  tolerating  sometimes  impertinent  questions, 
and  providing  additional  documentation  to  supplement  the   information  obtained 
on  site.     Their  leadership  stature  is  reflected  in  their  willingness  to  be 
of  service  to  this  study. 

The  role  of  the  Department  of  Health,  Education,  and  Welfare  and,   in  particular, 
the  Office  of  Quality  Standards  in  supporting  the  study  and  offering  advice 
is  gratefully  acknowledged. 

Most   important  were  contributions  of  the  staff  members  who  worked  under 
extremely  short  deadlines  to  produce  the  material   included   in  this  report. 
Special  thanks  go  to  Barbara  Buckley  for  administrative  assistance  throughout 
the  course  of  the  study  and  the  preparation  of  the   final  report. 

Linda  K.  Demlo,  Ph.D. 
Study  Director 
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Chapter  1 
SUMMARY  AND  RECOMMENDATIONS 


The  concept  of  quality  in  medical  care  historically  has  been  a  part  of  the 
ethos  of  the  medical  profession.     In  recent  years,  public   interest   in  health 
care  quality  and  cost  has  been  heightened  by  the  increase  in  public  expendi- 
tures for  health  care.     One  congressional  reflection  of  that  interest  was 
the  enactment   in  1972  of  legislation  that  authorized  the  establishment  of 
Professional  Standards  Review  Organizations  (PSROs)   [1]   to  monitor  the 
appropriateness  of  health  services  financed  by  the  Medicare,  Medicaid,  and 
Maternal  and  Child  Health  programs.     The  following  year,   Congress  requested 
a  major  study  of  alternative  mechanisms  for  health  care  quality  assurance,  a 
request  that  was  included  in  the  Health  Maintenance  Organization  Act  of 
1973.  [2] 

A  limited  version  of  that  study  was  contracted  to  the  Institute  of  Medicine 
by  the  Department  of  Health,  Education,   and  Welfare.     The  study,  reported 
in  this  document,  had  the  following  objectives: 

•  The  description  and  assessment  of  the  effect  of  operational  quality  re- 
view programs,  based  on  existing  written  information  and  supplemented 
by  observations  and  data  obtained  in  selected  site  visits; 

•  A  detailed  literature  review  of  several  topics,  designated  as  "priority 
areas"  because  of  their  importance   in  determining  the  effectiveness 

of  quality  assurance  programs  and  the  absence  of  reviews  that  integrate 
and  analyze  relevant  information; 

•  A  delineation  of  areas  in  which  additional  research  and  evaluation  are 
required. 

The  purpose  of  the  study  was  not  to  evaluate  the  PSRO  program — a  relatively 
recent,   large-scale  undertaking  not  yet  organizationally  complete.  Neverthe- 
less,  some  quality  assessment  programs  reviewed  were  PSROs,  and  some  recom- 
mendations refer  specifically  to  the  PSRO  program. 

In  limiting  the  scope  of  the  study,   the  steering  committee  established  cri- 
teria for  selecting  quality  assurance  programs  for  detailed  review.  Because 
the  concept  of  quality  is  multidimensional  and  complex,  a  single  definition 
of  quality  was  not  used.     Instead,  components  of  quality  that  have  been 
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identified  and  emphasized  by  existing  programs  were  accepted  within  the  focus 
of  this  study.     Particular  attention  was  given  to  programs  with  the  stated 
purpose  of  improving  the  health  status  and  satisfaction  of  patients.  The 
committee  did  specify  characteristics  of  an  ideal  quality  assurance  system: 
the  existence  of  an  organizational  entity  for  assessing  quality;  the  estab- 
lishment of  standards  or  criteria  against  which  quality  is  assessed;  a  routine 
system  for  gathering  information;  assurance  that  such  information  is  based 
on  a  representative  sample  of  the  total  population  of  patients  or  potential 
patients;  a  process  for  providing  the  results  of  review  to  patients,  the 
public,  providers,  and  sponsoring  organizations;  and  methods  for  instituting 
corrective  actions. 

A  survey  of  existing  programs,   including  many  of  those  reviewed  in  detail  in 
this  study,  makes  apparent  the  fact  that  most  programs  do  not  meet  all  of  the 
characteristics  of  an  ideal  system  specified  above.     Most  concentrate  on  the 
assessment  of  the  medical  care  process,  rather  than  the  assurance  of  improved 
quality  of  health  care.     Few  programs  routinely  provide  review  information  to 
patients  and  providers,   impose  corrective  actions,  or  determine  through  re- 
assessment whether  the  quality  of  care  has  improved.  Thus,  one  might  more 
realistically  describe  the  programs  as  quality  assessment  activities,  rather 
than  quality  assurance.     However,  the  term  "quality  assurance"  is  so  prevalent 
that  it  is  unlikely  to  be  obliterated  because  of  this  distinction.  Both  terms 
are  used  in  this  report. 

Several  timely  health  policy  issues  that   influence  the  quality  of  care  were 
excluded  from  this  report.     For  example,  malpractice  and  the  existence  of 
fraud  in  federally  financed  health  programs  were  not  considered.  Existing 
quality  assessment  programs  do  not  emphasize  the  detection  of  fraudulent 
practices  and  may  not  be  capable  of  doing  so.     Other  mechanisms  are  being 
developed  to  deal  with  malpractice.   Similarly,  many  factors   in  the  financing 
and  delivery  of  health  care  that  influence  quality  were  not  studied   in  detail. 
These  include  the  organizational  arrangements  through  which  care  is  provided, 
insurance  or  reimbursement  programs  that  specify  reimbursable  methods  of 
treatments,  health  professional  education,   the  numbers  and  distribution 
of  physicians  and  non-physicians  geographically  and  among  specialities,  provi- 
sions  for  licensing  and  certification  of  individual  health  care  providers 
and  facilities,  and  the  recording,  storage,  and  retrieval  of  information 
about  patients. 

Thus,  the  study  is  not  an  exhaustive  review  of  all  factors   in  the  health  care 
sector  that  influence  the  quality  of  care.     Rather,   it   is  an  examination  of 
existing  quality  review  programs.     The  primary  purpose   is  to  describe  the 
manner  in  which  they  function  and  their  reported  effectiveness  in  improving 
health  status  or  patient  satisfaction  and  conserving  resources.  Additional 
issues  relating  to  quality  assurance  are  reviewed  in  the  priority  areas: 
outcome-oriented  approaches  to  quality  assurance,  quality  assurance  for  ambula- 
tory care,   quality  assurance  for  long-term  care,  methods   for  changing  behavior 
patterns  of  health  care  providers,  and  patient  and  consumer  involvement  in 
quality  assurance  programs. 

Determining  the  effectiveness  of  quality  review  programs  is  particularly  dif- 
ficult— in  part  because  of  methodological  problems,  but  also  because  of  the 


3 


absence  of  valid  and  reliable  information.     The  effect  of  review  on  quality 
of  care  usually  is  described  anecdotally  because  of  limitations  in  current 
measures  of  health  status  or  outcome.     Although  improvements   in  quality  at- 
tributed by  program  officials  to  review  may  be  impressive,  there  is  no  cur- 
rent method  for  relating  the  individual  anecdotes  to  the  total  review  effort 
in  a  manner  which  would  facilitate  the  determination  of  cost  effectiveness. 
Because  of  the  close  link  between  quality  and  utilization  of  services,  the 
relative  ease  of  measuring  utilization,  and  its  associated  relationship  to 
cost  control,  effectiveness  is   frequently  measured  in  terms  of  utilization, 
as  expressed  in  costs.     However,  the  measures  customarily  used  are  frequently 
inadequate  and  difficult  to  interpret.  This  is  discussed  in  detail  in  the 
body  of  the  report.     Variations  among  review  programs  limit  one's  ability 
to  compare  program  effectiveness.     The  relative  effect  of  different  types 
of  review  mechanisms  might  be  assessed  by  comparing  programs  according  to 
the  magnitude  of  changes  within  them,  assuming  all  other  influences  could 
be  held  constant.     However,  adequate  baseline  data  are  not  available  to  per- 
mit before-and-af ter  measurements.     A  period  of  time  must  be  allowed  for  pro- 
gram development  and  refinement  before  effectiveness  can  realistically  be 
expected. 


DESCRIPTION  OF  QUALITY  ASSESSMENT  PROGRAMS 

The  18  quality  assessment  programs  reviewed  in  detail  represent  what  have  been 
regarded  as  the  "better"  programs.     Most  have  benefitted  from  considerable 
financial  support  and  extensive  experience,  as  compared  with  programs  not 
included  in  the  study.     The  programs  reviewed  cannot  be  regarded  as  a  repre- 
sentative national  sample.     Many  of  them  were  pioneers  in  the  field  of  quality 
assurance  and  have  created  models  that  have  been  adapted  elsewhere.  Their 
accomplishments,  however,  were  not  made  without  difficulties,   and  these  may 
be  equally  instructive.     To  draw  conclusions  from  the  experiences  of  these 
programs  perhaps  would  be  to  pass  judgment  prematurely  on  programs  that  have 
had  insufficient  time  to  work  on  a  very  complex  problem.     The  information  pre- 
sented here,  therefore,   is  intended  to  assist   in  re-examining  and,  possibly, 
redirecting  current  efforts  before  they  become  so  established  in  custom  as  to 
make  modification  difficult. 

A  majority  of  the  programs  visited  can  be  characterized  as  follows: 

•  The  stated  goals  are  to  ensure  high  quality  medical  care  at  a  reasonable 
cost.     But  the  goals  are  not  expressed  in  terms  that  permit  measurement  of 
the  degree  to  which  they  are  achieved.     The  margin  by  which  quality  might 
be  improved  is  not  known.     Even  rough  estimates  of  the  magnitude  of  cur- 
rently inappropriate  care  were  unavailable.     Without  such  measures,  it 
becomes  difficult  to  determine  whether  a  program  is  achieving  its  objec- 
tives and  whether  the  resultant  improvement  is  sufficient  to  justify 
program  expenditures. 

•  Programs  are  oriented  toward  users  of  health  services,  rather  than  people 
who  do  not  use  services,  and  with  a  few  exceptions,  do  not  consider 
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access  to  care  and  under-utilizat ion  of  services. 

•  Compliance  with  PSRO  review  requirements  is  the  major  concern  of  most 
hospital  programs.     PSRO  review  components  include:     concurrent  review — 
intended  to  assure  that  individual  hospital  admissions  and  continued  stays 
are  medically  necessary;  medical  care  evaluations  (MCEs)  or  medical  audits — 
a  detailed,   frequently  retrospective  review  of  the  quality  of  care  given 

to  groups  of  patients;  and  profile  analysis — a  retrospective  analysis  of 
patterns  of  care  that  may  concentrate  on  particular  diagnoses,  patients, 
or  physicians  and  identify  areas  for  special  attention  by  either  concur- 
rent review  or  MCEs. 

•  Most  hospital  review  programs  place  primary  emphasis  on  concurrent  review 
activities.     There  is  considerable  variation  among  programs  in  the  timing, 
depth,  and  frequency  of  review.     Similarly,   the  degree  to  which  review 
coordinators  and  physician  advisors  are  trained  and  supervised  varies  by 
program. 

•  Less  emphasis  is  placed  on  medical  care  evaluation  studies.     Common  prob- 
lems have  been  encountered  in  conducting  MCEs:     the  incompatibility  between 
requirements  of  the  Joint  Commission  on  Accreditation  of  Hospitals  (JCAH) 
and  PSRO;  the  difficulty  of  selecting  audit  topics  which  result  in  the 
identification  of  significant  problems  so  that  improvements  can  be  made; 
the  difficulty  of  developing  criteria  that  are  relevant  for  all  patients 
without  becoming  too  general;  and  the  difficulty  of  achieving  change,  once 
deficiencies  in  patient  care  have  been  identified.     Innovative  efforts 
which  address  these  problems  are  underway. 

•  Profile  analysis,  the  third  component  of  PSRO  review,   is  the  least 
developed . 

•  Integration  of  the  three  review  components  is  seldom  achieved  within 
hospitals  or  within  PSRO  administrative  staffs. 

•  The  most  common  type  of  quality  assessment   for  ambulatory  care   is  based  on 
a  review  of  claim  forms  submitted  by  physicians   to   fiscal  intermediaries 
for  reimbursement  purposes.     Alternatives  to  claims  review  should  permit 
greater  emphasis  on  quality  by  reviewing  the  provision  of  medical  care 
over  time  and  assessing  access  to  care  and  health  outcomes.  However, 
they  require  further  refinement  and  evaluation  before  being  widely  imple- 
mented . 

•  There  is  difficulty  in  achieving  improvements  after  deficiencies  in 
patient  care  have  been  documented.     Most  programs  rely  on  educational 
methods  for  encouraging  improved  performance.     Some  internal  appraisals 
of  the  effects  of  review  have  been  made,  but  no  program  has  established 
a  formal  mechanism  for  self-assessment.     Some  have  been  evaluated  by 
external  groups,  however. 


5 


EFFECTIVENESS  OF  QUALITY  ASSESSMENT  PROGRAMS 

The  steering  committee  believes  that  the  widespread  interest  in  quality  assur- 
ance activities  and  the  intellectual  stimulation  and  professional  re-examination 
that  occur  as  programs  are  initiated  and  standards  for  care  are  established 
should  eventually  improve  the  general  quality  of  medical  practice.     The  commit- 
tee found  impressive  examples  of  stated  improvements   in  quality  and  changes 
in  utilization  of  health  care  services  in  the  programs  reviewed.  Assessing 
the  broader  impact,  however,   requires  consideration  of  the  total  magnitude 
of  the  effort,   not  merely  isolated  examples. 

The  difficulty  of  measuring  the  quality  of  care  and  the  effectiveness  of 
quality  review  systems  was  noted  at  the  outset.     Information  on  cost  and  effec- 
tiveness,  in  particular,  covers  a  relatively  short  time  span  and  reflects  what- 
ever information  was  readily  available  rather  than  what  could  be  collected  in 
a  carefully  designed  effort  at  evaluation.     These  limitations  notwithstanding, 
the  steering  committee  was  able  to  reach  some  preliminary  conclusions  about 
the  current  effectiveness  of  programs  visited: 

•  Existing  information  does  not  substantiate  the  effectiveness  of  MCEs . 
MCEs  or  medical  audits  have  been  required  for  accreditation  and  reimburse- 
ment purposes  for  several  years.     Yet,   there  is  no  reliable  source  of  data 
to  reflect  the  numbers,   topics,   and  associated  costs  of  currently  performed 
MCEs,   the  identified  deficiencies  in  patient  care,   the  remedial  actions 
proposed  and  taken,  or  the  extent  and  duration  of  improvements  in  patient 
care.     MCEs  may  have  caused  improvements  in  quality,  but  reliable,  general- 
izable  assessments  are  not  available. 

•    Evidence  is  not  yet  available  for  a  conclusion  that  hospital  concurrent 
review  programs  are  effective.     Although  changes   in  utilization  patterns 
have  been  noted,   the  reasons  are  not  adequately  understood.     The  costs 
of  conducting  concurrent  review  vary  widely.     Assertions  of  cost  savings 
are  exaggerated,  because  they  assume  that  total  per  diem  cost  will  be 
saved  for  each  day  of  care  denied  and  do  not  adequately  take  into  account 
fixed  hospital  costs  or  the  cost  of  alternative  care. 

•  Most  ambulatory  care  claims  review  programs  considered  in  this  study  yield 
dollar  reductions  in  submitted  claims  that  are  more  than  adequate  to  pay 
the  costs  of  review,   and  some  improvements  in  quality  have  been  noted. 

At  least  for  the  fiscal   intermediary,  claims  review  is  cost  effective. 
A  claim  denied  or  reduced,  however,   is  not  necessarily  a  claim  unpaid — 
some  providers  are  persistent  in  recovering  some  portion  of  their  fee, 
which  may  be  eventually  paid  by  the  patient,  other  fiscal  intermediaries, 
or  society.       Furthermore,  most  savings  come  from  administrative  reviews 
of  patient  eligibility  for  insurance  coverage,   the  range  of  reimbursable 
benefits,  or  the  amount  of  reimbursement  claimed.     These  savings  would 
be  realized  under  most  claims  review  systems  and  generally  are  unrelated 
to  considerations  of  either  quality  or  appropriateness  of  care.  The 
additional  benefits  from  the  medical  peer  review  component  of  claims 
review  are  not  well  documented. 
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The  reasons  for  lack  of  demonstrated  effectiveness  of  quality  review  programs 
are  difficult  to  isolate.     Whether  they  are  due  to  faulty  conceptualization 
of  the  nature  of  quality  or  to  defects  in  program  design  is  not  clear.  At 
the  very  least,   it   is  evident  that  the  objectives  of  current  programs  are 
not  well  specified  and  include  a  mixture  of  goals:     cost  control  alone, 
utilization  control   (a  desire  to  increase  the  potential  benefits  of  care  by 
controlling  the  types  and  quality  of  resources  used,  which  also  generate 
costs),   and  increased  effectiveness  of  medical  care  (improved  quality  in 
terms  of  greater  patient  satisfaction  and  better  health  outcome).  Though 
these  objectives  may  be  conceptually  interrelated,   it  is  not  clear  that 
a  single  review  program  can  address  these  and  possibly  other  objectives 
simultaneously. 

These  uncertainties  notwithstanding,   the  steering  committee  believes  that 
some  monitoring,   perhaps  on  a  sample  basis,   of  the  quality  of  medical  care 
provided  to  all  patients — not  just  those  for  whom  the  federal  government  has 
financial  responsibility — is  essential.     At  the  level  of  quality  assurance 
expenditures  anticipated  for  FY  1977,   the  total  cost  for  hospital  and  ambula- 
tory review  could  exceed  $1,250,000,000  annually  if  extrapolated  for  the  entire 
U.S.   population.     There  is  a  need  for  less  expensive  methods  to  achieve  better 
results.     It  should  be  possible  to  increase  the  efficiency  of  existing  quality 
assessment  techniques  while  also  developing  new  strategies   for  areas  which 
do  not  now  receive  adequate  attention. 

The  steering  committee  identified  a  number  of  actions  that  could  be  taken 
to  achieve  these  goals.     The  committee's  recommendations  are  divided  into 
two  sections.     General  recommendations  stem  primarily  from  the  site  visits 
and  are  not  confined  to  any  specific  priority  area.     They  are  mainly  actions 
which  should  be  taken  by  national  policymakers,   although  other  national 
policy  recommendations  in  the  priority  areas  are  properly  cited  under  those 
headings,  which  follow  the  general  recommendations. 

Both  general  and  priority  area  recommendations  are  categorized  into  those 
which  should  be  implemented  immediately  and  long-term  recommendations  which 
require  further  research  and  evaluation.     Many  of  the  long-term  recommenda- 
tions emphasize  the  need  to  develop  more  refined  techniques  for  conducting 
quality  assurance  activities,  with  particular  concentration  in  two  areas: 
research  to  develop  more  reliable  and  valid  assessment  tools  to  measure 
the  levels  of  quality;   and  research  to  foster  the  improvement  or  assurance 
of  quality,  which  involves  the  development  of  better  methods  for  altering 
the  behavior  of  both  health  care  providers  and  consumers.     Thus,  many  of 
the  recommendations  for  the  priority  areas  constitute  a  long-range  research 
agenda . 

GENERAL  POLICY  RECOMMENDATIONS 
Immediate 

1.   Review  techniques  should  be  refined  to  facilitate  a  more  concentrated 
(targeted)  examination  of  diagnoses,   patients,   or  providers  associated 
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with  questionable  patterns  of  care,   as  an  alternative  to  the  current  practice 
of  reviewing  all  cases.     A  targeted  review  should  enable  more  frequent 
identification  of  truly  inappropriate  care  and  less  frequent  review  of 
appropriate  care,  which  should  increase  the  efficiency  of  review.  Cases 
excepted  from  routine  review  should  be  monitored  periodically  to  assure  that 
more  frequent  review  is  not  warranted.     Profile  analysis  must  be  further  de- 
veloped to  provide  the  information  to  identify  patients  or  providers  who 
require  more  concentrated  review. 

2.  Criteria  for  excepting  cases  from  review  should  be  clearly  specified. 
The  new  Medicare  and  Medicaid  utilization  review  regulations,  which  permit 
each  hospital  independently  to  specify  cases  that  will  not  be  routinely  re- 
viewed and  do  not  include  adequate  provisions  for  monitoring,  may  reduce 
the  likelihood  of  identifying  inappropriate  care. 

3.  Within  PSROs,  a  conscious  effort  is  needed  to  integrate  the  three  types 
of  review.     The  common  practice  of  delegating  responsibility  to  hospitals 
for  conducting  MCEs  independent  of  concurrent  review  should  be  discontinued 
because  it  encourages  fragmentation.     Within  hospitals,  PSRO  review  activities 
should  be  better  integrated  with  prior  utilization  review  and  other  quality 
assurance  activities. 

4.  Intensified  efforts  should  begin  immediately  to  evaluate  both  federal 
and  privately  sponsored  health  care  quality  assurance  systems  by  comparing 
the  quality  of  care  in  geographic  areas  with  and  without  quality  review 
programs,  or  in  areas  of  otherwise  similar  characteristics  but  different 
types  of  review.     This  should  have  been  accomplished  before  a  uniform 
national  quality  assurance  program  was  required  by  the  PSRO  legislation. 
Nevertheless,   the  committee  believes  that  planned  experimentation  should 
still  be  possible  in  order  to  determine  the  relative  effects  of  alternative 
review  mechanisms  on  health  status,  utilization  and  cost  of  services,  and 
other  measures  of  quality. 

5.  There  should  be  fewer,  better  designed,   and  better  evaluated  MCEs.  The 
JCAH  and  PSRO  requirements  for  MCEs  should  be  compatible  in  content,  as 
well  as  numerical  requirements.     Hospitals  should  be  permitted  to  count 
re-audits  of  completed  audits  in  fulfilling  MCE  numerical  requirements. 

6.  Criteria  should  be  developed  for  categorizing  successful  and  unsuccess- 
ful MCEs  and  isolating  factors  associated  with  success,   so  that  more  effec- 
tive MCEs  may  evolve.     Data  bases  must  be  developed  to  describe  current  MCEs, 
so  that  a  more  definitive,   future  assessment  of  effectiveness  can  be  made. 
Since  there  currently  is  no  "good"  model  of  a  MCE,  a  wide  range  of  innova- 
tion and  evaluation  should  be  encouraged.     The  effectiveness  of  concurrent 
and  prospective  MCEs,   that  permit  direct  intervention  in  the  process  of  care 
where  warranted,   should  be  tested.     The  relative  merits  of  areawide  MCEs,  as 
opposed  to  individual  hospital  MCEs,   should  be  assessed. 

7.  Uniform  data  elements,  but  not  necessarily  data  formats,   should  be  re- 
quired in  all  health  care  settings  to  facilitate  quality  assurance  activities, 
as  well  as  program  management,   planning,   and  evaluation.     Requirements  for 
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the  Uniform  Hospital  Discharge  Data  Set  as  modified  for  PSROs  should  be 
enforced.     The  Mininum  Ambulatory  and  Long-Term  Care  Data  Sets  should  be 
implemented.     Methods  for  linking  information  from  the  Medicare  Part  B 
supplementary  insurance  program  with  Part  A  hospital  information  should  be 
devised.     More  general  methods  are  needed  to  integrate  hospital  and  ambula- 
tory patient  care  information  using  a  common  identification  number.  Better 
"denominator"  data  must  be  generated  to  define  the  population  eligible  for 
care  and  to  provide  the  basis  for  monitoring  utilization  of  services.  Im- 
portant confidentiality  issues  must  be  resolved  to  protect  individual  privacy 
and  the  public  right  to  information. 

8.  Both  nationally  and  locally,  PSROs  and  Health  Systems  Agencies  (HSAs) 
should  establish  mutually  beneficial  working  relationships,  beginning  with  an 
exchange  of  data.     HSAs  have  information  on  the  population  eligible  for  care, 
which  is  needed  by  PSROs.     PSROs  can  document  variations  in  the  use  of 
services,  which  may  suggest  problems  in  access  and  under-utilization  that 
should  be  addressed  by  both  HSAs  and  PSROs. 

9.  Quality  assurance  programs  should  further  specify  their  objectives  and 
establish  internal  self-assessment  units  for  program  evaluation.  The 
Department  of  Health,  Education,   and  Welfare  should  provide  technical  assis- 
tance involving  both  trained  researchers  and  persons  experienced  in  peer 
review.     Appropriate  links  with  health  services  research  centers  should  be 
established.     Additional  support  is  required  for  research  training  programs 
to  develop  the  necessary  cadre  of  skilled  personnel. 

10.  Policy  mandates  for  quality  assurance  should  impose  comparable  levels 

of  stringency  on  all  health  care  delivery  arrangements,  even  though  the  manner 
in  which  requirements  are  met  may  vary.     The  greater  ease  of  conducting  quality 
assessment  activities  in  larger,   formally  organized  health  care  programs,  such 
as  Health  Maintenance  Organizations,   should  not  lead  to  the  imposition  of  more 
rigorous  requirements  on  such  organizations. 

GENERAL  POLICY  RECOMMENDATIONS 
Long-Term 

1.  Better  evaluative  measures  are  needed  to  identify  and  aggregate  the 
effects  on  health  status  that  result  from  the  provision  of  medical  care, 
and  to  assess  the  impact  of  quality  assurance  programs,  continuing  educa- 
tion,  and  other  activities  designed  to  improve  the  quality  of  care. 

2.  Better  techniques  are  needed  to  determine  the  effect  of  quality  as- 
surance programs  on  utilization  and  cost  of  medical  care.     Data  should  be 
adjusted  to  enable  comparisons  among  facilities  with  different  patient  and 
provider  characteristics.     Measures  should  assess  the  effect  of  review  on 
the  total  community,   rather  than  individual  facilities,   and  should  take 
into  account  the  costs  of  alternative  care.     Additional  research  is  needed 
to  determine  the  conditions  under  which  hospital  costs  vary  according  to 
occupancy  rate  in  both  short  and  long-term  situations.     Adjustments  for 
fixed  costs  and  the  cost  of  alternative  care  should  be  included  in  esti- 
mates of  cost  savings  resulting  from  utilization  review. 
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3.  The  assumption  that  hospitals  have  incentives  to  conduct  meaningful  util- 
ization review  programs  should  be  examined.  Current  reimbursement  mechanisms 
provide  little  incentive  for  hospital  administrators  to  reduce  variable  costs 
associated  with  lower  hospital  utilization.     Therefore,  actual  savings  may 

be  minimal. 

4.  The  indices  of  quality  currently  used  by  quality  assurance  programs 
should  be  expanded.     Access  to  care  and  potential  under-utilization  should 

be  assessed.     This  requires  attention  to  the  availability  of  health  care  pro- 
viders and  facilities,  appropriate  links  between  levels  of  care,  and  policies 
(such  as  on-call  arrangements)  to  assure  that  services  are  easily  accessible. 
Benefit  packages  and  reimbursement  policies  should  be  examined  to  determine 
their  impact  on  the  quality  of  care,  particularly  with  respect  to  coverage 
of  long-term  care,  both  institutional  and  non-institutional.     Greater  efforts 
should  be  made  to  link  the  separate  programs,  which  currently  address  only 
care  provided  to  specific  patients  by  specific  providers  or  facilities,  into 
an  integrated  program  which  assesses  the  quality  of  care  provided  by  the 
total  health  system. 

5.  A  systematic  accumulation  of  data  is  needed  to  describe  current  patterns 
of  medical  care  in  all  settings.     Special  attention  should  be  given  to  unusual 
departures  from  customary  practice,  the  extremes  of  under-  and  over-utilization, 
and  the  reasons  for  such  variation.     This  information  should  provide  a  better 
estimate  of  the  margin  by  which  quality  and  utilization  of  services  might  be 
improved,  which  in  turn,  would  help  to  determine  the  magnitude  of  the  re- 
quired quality  assurance  effort  and  identify  areas  of  achievable  gain  for 
special  attention. 

6.  The  curricula  for  health  professionals  should  include  courses  in  health 
care  evaluation  and  assessment  designed  to  be  relevant   in  routine  practice 
and  implemented  throughout  one's  professional  career.     Quality  assurance  ac- 
tivities should  be  applied  in  facilities  where  physicians-in-training  provide 
care . 


RECOMMENDATIONS  FOR  ASSESSING  HEALTH  OUTCOMES 
Immediate 


1.     Additional  research  is  needed  to  develop  substitute  or  short-term  outcome 
measures  which  occur  closer  in  time  to  the  provision  of  care  than  final  end- 
result  measures.     The  relationship  between  such  measures  and  process  and 
final  outcome  measures  should  be  established. 

Long-Term 

1.     The  steering  committee  believes  that  health  care  should  be  assessed  on 
the  basis  of  health  outcome,  despite  the  limitations  of  current  measures 
and  uncertainties  about  the  contribution  of  medical  care  to  health  status. 
Patient  satisfaction  must  be  recognized  as  one  indicator  of  outcome. 
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Research  is  needed  to  develop  better  measures  of  patient  satisfaction  and 
health  status.     In  addition  to  the  use  of  short-term  outcome  measures, 
greater  reliance  should  be  placed  on  existing  instruments   for  assessing 
functional  status  and  the  growing  body  of  knowledge  of  the  natural  history 
of  disease — particularly  for  assessing  the  progress  of  the  chronically 
ill. 

2.  In  the  same  sense  that  process  measures  of  quality  are  currently  required 
of  PSROs ,   limited  post-discharge  outcome   information  should  be  gathered.  The 
cost  and  utility  of  outcome  studies  should  be  carefully  monitored.  Outcome 
data  should  assist  in  identifying  patients  and  providers  for  whom  the  process 
of  care  should  be  more  thoroughly  assessed  and  in  isolating  areas  in  which 
efficacy  studies  are  required.     The  accumulated  data  should  lead  to  a  better 
understanding  of  the  natural  course  of  illness.     Over  time,  sufficient  knowl- 
edge should  be  accumulated  so  that  if  patients  of  a  particular  provider  have 
not  progressed  as  expected,  the  provider's  treatment  methods  could  be  ques- 
tioned or  the  patient  referred  elsewhere  for  evaluation  and  consultation. 

3.  Individual  practitioners  should  be  encouraged  to  join  with  their  patients 
in  establishing  outcome  objectives  for  patient  care  and  examining  reasons  for 
failure  to  meet  them. 

4.  Additional  research  is  needed  to  establish  the  natural  history  of  diseases 
and  the  efficacy  of  medical  procedures  and  therapies.     For  research  findings 
to  be  useful  in  assessing  the  quality  of  care,  determinations  of  efficacy 
should  be  made  under  average  as  well  as  ideal  treatment  situations  at  various 
points  in  time  and  should  include  a  broad  range  of  outcome  measures.  The 
Department  of  Health,   Education,   and  Welfare  should  further  specify  the  re- 
sponsibilities of  its  component  agencies  in  this  area  and  increase  the  level  of 
funding. 

RECOMMENDATIONS  FOR  AMBULATORY  QUALITY  ASSURANCE 
Immediate 

1.  Ambulatory  claims  review  should  be  more  widely  implemented  in  an  experi- 
mental manner  while  more  appropriate  ambulatory  quality  assurance  techniques 
are  being  developed.     Despite  the  limitations  of  claims  review,   it  will  per- 
mit the  detection  of  the  most  serious  deficiencies.     Governmental  agencies 
and  other  purchasers  of  health  care  should  be  encouraged  to  require  more 
stringent  claims  review  by  their  fiscal  intermediaries.     Careful  evaluation 
of  these  programs  should  be  required. 

2.  Closer  monitoring  should  begin  immediately  of  pharmacy  services,  small 
clinical  laboratories,  and  free-standing  radiological  units.  Monitoring 
techniques  using  pre-ident i f ied  specimens  should  be  more  widely  applied  to 
determine  the  accuracy  of  judgments  within  labs  and  radiology  services. 
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Long-Term 

1.  Intensified  research  and  development  is  needed  for  ambulatory  quality 
assessment  methods.     Primary  ambulatory  care  is  different   from  secondary  and 
tertiary  care  and  requires  different  quality  assessment  techniques.  Many 
ambulatory  review  programs  rely  on  a  diagnostic-specific  review  of  the  medical 
record,  but  the  bulk  of  primary  ambulatory  care  consists  of  signs  and  symp- 
toms that  cannot  readily  be  assigned  to  diagnostic  categories.  Classification 
schemes  to  record  patient-reported  symptoms  are  being  developed  and  could  form 
the  basis  of  an  experimental  quality  assurance  project.     Another  approach  might 
focus  on  the  basic  skills  or  tasks  which  constitute  primary  ambulatory  care, 
such  as  the  elicitation  of  signs  and  symptoms  and  their  history,  performance 

of  a  physical  exam,  the  synthesis  of  this  information  into  recommendations  for 
care,  and  determination  of  the  appropriate  point  for  referral.  Much  of  the 
success  of  primary  care  depends  on  the  extent  to  which  the  practitioner  co- 
ordinates care  provided  over  a  relatively  long  period  of  time.  Very  little 
of  this  information  is  found  in  the  medical  record,  and  other  recording  and 
assessment  methods  must  be  devised. 

2.  Additional  work  is  needed  to  devise  means  for  supplementing  information 
recorded  on  the  claim  form.     The  Minimum  Ambulatory  Care  Data  Set  should  be 
the  basis  on  which  such  work  proceeds.     In  addition,  the  value  of  diagnostic, 
patient,  and  laboratory  registries  to  facilitate  problem  identification  and 
provide  information  over  time  should  be  explored.     Probability  sampling  tech- 
niques must  be  developed  for  claims  review  to  focus  on  patients  and  providers 
who  fall  at  the  extremes  of  distributions  of  care  patterns  and  at  the  same 
time  give  estimates  of  the  broader  spectrum  of  care  provided  to  the  total 
population. 

3.  A  single  approach  to  quality  assurance  will  not  accommodate  the  diversity 
of  functions  and  personnel   included  within  the  ambulatory  care  sector;  further 
research  is  required  before  a  range  of  proved  alternative  methods  can  emerge. 
The  widest  possible  range  of  review  techniques  should  be   included  in  the  am- 
bulatory demonstration  projects  to  be  funded  by  the  Bureau  of  Quality 
Assurance.     Recipients  of  awards  should  not  necessarily  be  limited  to  PSROs . 
The  budget  for  these  activities  should  be  increased. 

RECOMMENDATIONS  FOR  LONG-TERM  CARE 
Immediate 

1.  Existing  standards  to  protect  residents  of  long-term  care  facilities 
should  be  enforced,  while  more  appropriate  quality  assurance  mechanisms 
for  long-term  care  are  being  developed. 

2.  The  certification  and  licensure  process  for  long-term  care  providers 
should  be  reconsidered.     The  Department  of  Health,  Education,  and  Welfare 
study,   scheduled  to  begin  in  April  of  1977,   should  go  beyond  a  review  of 
existing  structural  standards  to  address  more  fundamental   issues  of  quality 
and  analyze  the  financial  and  other  ramifications  of  forced  compliance 
with  standards. 
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Long-Term 

1.  Quality  assurance  programs  for  long-term  care  should  be  designed  to 
address  the  unique  needs  of  the  chronically  ill.     The  etiology  of  many 
chronic  conditions  remains  obscure;  many  individuals,  particularly  the  aged, 
have  several  chronic  conditions.     Thus,  an  assessment  of  quality  based 

on  diagnostic-specific  criteria  is  often  inappropriate,  and  functional  status 
is  a  more  relevant  measure.     Furthermore,  because  of  the  long-term  nature 
of  the  patient's  condition  and  frequent   fluctuations  in  physical  and  mental 
states,   treatment  requirements  vary.     Patients  may  require  differing  levels 
of  care  within  a  relatively  short  time  period,  ranging  from  intensive  hospital 
care,   skilled  nursing  services,  custodial  care,  or  home  health  services, 
to  periodic  office  visits.     Methods  for  assessing  the  quality  of  care  should 
include  all   sources  of  care  and  should  consider  the  impact  of  care  on  the 
patient's  expected  and  actual  ability  to  function  in  daily  life. 

2.  The  responsibility  for  quality  assurance   in  long-term  care  belongs  at 
the  community  level  so  that  an  integrated  review  of  the  total  range  of  ser- 
vices can  occur.     Anything  less  will  be  based  on  evaluation  of  care  from  the 
fragmented  view  of  individual   facilities  or  programs  and  will  perpetuate  the 
inefficient  and  costly  services  which  currently  exist.     Steps  should  be  taken 
to  develop  community-level  organizations  to  include  a  broad  range  of  providers, 
facilities,  professional  groups,  consumers,   and  representatives  from  planning 
and  certifying  agencies.     The  community  organization  should  consider  such 
issues  as  access  to  care,  appropriateness  of  placement,   scope  of  available 
services,  and  the  accumulation  of  uniform  data.     Assessment  of  the  technical 
components  of  care  could  be  delegated  to  PSROs  and  other  groups  of  health 

care  providers.     Demonstrations  should  be  initiated  to  test  the  feasibility 
of  such  an  approach  in  terms  of  both  cost  and  effectiveness.  Evaluation 
should  occur  after  prototype  organizations  have  passed  the  developmental 
phase . 

3.  State  and  federal  reimbursement  policies  for  long-term  care  should  be 
reformed.     State  and  federal  regulations   for  reimbursement  and  accounting 
should  be  made  compatible  and  redesigned  to  enhance  their  influence  on  the 
quality  of  care.     The  levels  of  reimbursement  should  not  be  so  inadequate  as 
to  lead  to  poor  quality.     Experimental  reimbursement  projects  should  examine 
the  effect  of  capitation,  which  would  permit  the   individual   to  move  from  one 
level  of  care  to  another  without  being  penalized.     Similarly,  experimenta- 
tion with  facility  reimbursement  rates  based  on  the  customary  mix  of  patients, 
rather  than  specific  patients,  might  permit  patients  to  be  moved   from  one 
level  of  care  to  another,  depending  on  their  conditions. 

4.  Support  of  existing  programs  to  train  personnel  for  work  in  long-term  care 
should  be  continued  and  expanded.     Program  content  should  focus  on  the  unique 
characteristics  of  long-term  care,  the  multiplicity  of  skills  which  are  re- 
quired to  meet  patient  needs,  and  the  necessity  of  a  team  approach. 

5.  The  long-term  care  quality  assurance  demonstration  projects  to  be  funded  by 
the  Bureau  of  Quality  Assurance  should  represent  a  wide  variety  of  alternative 
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approaches  to  review  and  should  not  be  limited  to  PSROs.     The  budget  for 
these  activities  should  be  protected  and  expanded. 

RECOMMENDATIONS  FOR  IMPROVING  PROVIDER  PERFORMANCE 
Immediate 

1.  There  should  be  no  mandated  provisions  for  any  specific  technique  for 
improving  provider  performance  in  the  immediate  future,   including  continuing 
medical  education.     Existing  evidence  of  effectiveness  is  inadequate. 

2.  Because  of  the  limitations  of  education  in  improving  individual  provider 
performance,   alternative  methods  should  be  explored.     In  particular,  the 
influence  of  the  organization  of  health  care  resources  on  quality  needs 
immediate  attention. 

Long-Term 

1.  Research  is  needed  to  devise  methods  for  encouraging  improvements  in 
patient  care,  once  deficiencies  are  identified.     Relevant  literature  from 
the  social  sciences,  as  well  as  from  medical  education,   should  be  utilized. 
Existing  quality  deficiencies  should  be  categorized  to  assist  in  determining 
the  reasons  for  their  occurrence  and  the  design  of  appropriate  corrective 
actions.     All  methods  for  improvement  should  be  carefully  evaluated  to  deter- 
mine the  extent  to  which  they  result  in  lasting  behavior  change. 

2.  Studies  are  needed  of  the  effectiveness  of  various  methods  for  informing 
a  physician  that  he  is  providing  inadequate  care,   including  presentation  of 
information  describing  his  practice  patterns  compared  with  his  peers,  structur- 
ing the  information  to  emphasize  particular  deficiencies,  or  providing  incen- 
tives for  review  and  change.     Reasons  for  failure  to  change  should  be  explored. 

3.  For  instances  in  which  clearly  inappropriate  care  is  identified  and  be- 
havior does  not  change,   sanctions  which  are  less  drastic  than  permanent  loss 
of  licensure  may  be  more  readily  applied.     Experimentation  is  needed  with 
intermediate  sanctions,   including  curtailment  of  privileges,  licensing 

with  restrictions  on  specified  areas  of  practice,  mandatory  supervision 
of  medical  practice,  or  remedial  education.     Demonstrations  should  test 
the  effectiveness  of  equipping  PSROs  with  a  wider  range  of  sanctions  for 
clearly  inappropriate  behavior,   including  more  direct  links  with  licensing 
bodies  or  authorizing  the  PSRO  to  remove  a  license  with  due  cause.  State 
legislative  bodies  should  waive  or  amend  existing  statutes,   if  necessary, 
to  permit  such  experimentation. 

4.  The  feasibility  and  effectiveness  of  publicizing  instances  of  persistently 
poor  quality  care  by  individual  practitioners  in  public  media  should  be 
explored . 
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RECOMMENDATIONS  FOR  CONSUMER  INVOLVEMENT 


Immediate 


1.     Health  care  consumers,  both  individually  and  collectively,  should  be 
educated  to  accept  greater  responsibility  for  their  own  health  and  should 
be  involved  in  decisions  regarding  the  provision  and  evaluation  of  health 
care.     One  immediate  step  for  furthering  this  concept  would  be  the  inclu- 
sion of  representatives  of  the  public  as  members  of  the  National  Professional 
Standards  Review  Council.     This  may  require  a  legislative  amendment. 


Long-Term 

1.     The  steering  committee  believes  that  consumer  involvement  in  the  planning, 
management,  and  evaluation  of  health  care  programs  should  be  encouraged  and 
expanded.     A  better  public  understanding  of  the  determinants  of  health,  the 
limitations  of  health  care,  the  resources  required  to  provide  it,  and  the  ne- 
cessity to  work  in  partnership  with  professionals  to  create  a  system  of  health 
care  should  result  in  improvements  in  the  quality  and  appropriateness  of  health 
services  and  a  healthier  public.     The  objective  is  clear,  but  the  methods  for 
achieving  it  are  not.     Research  and  experimentation  are  required. 


2.  More  information  is  needed  to  identify  aspects  of  health  care  that  are 
important  to  the  consumer,  which  can  then  be   incorporated  into  valid  and 
reliable  instruments  for  assessing  patient  expectations  and  satisfaction. 
Additional   information  also  is  needed  to  relate  expectations  and  satisfaction 
to  compliance  with  medical   instructions  and  to  health  outcome.  Once  the 
measures  are  adequate,  the   feasibility  of  implementing  them  in  formal  quality 
assurance  programs  can  be  better  tested. 

3.  Although  some  health  education  programs  have  been  effective  in  changing 
patient  behavior,  additional  research  is  needed  to  identify  factors  associated 
with  effectiveness.     Attention  should  be  given  to  the  effect  of  alternative 
media,  differing  levels  of  patient  and  family  involvement,  the  duration  of 
behavior  change  and  whether  reinforcement  is  needed,  the  potential  contribu- 
tion of  motivational  research,  and  patient  factors  which  may  influence  effec- 
tiveness, such  as  emotional  state,  demographic  characteristics,  and  health 
status.     Different  approaches  may  be  required  for  different  patient  condi- 
tions, ranging  from  preventive  care  to  acute  illnesses  and  chronic  condi- 
tions . 


4.     When  patient  education  is  known  to  be  essential,  quality  assessment 
criteria  should  require  that  education  activities  be  performed.     To  the  extent 
that  process-oriented  criteria  are  used  to  monitor  care,  current  efforts  to 
include  educational  components  (such  as  dietary  instruction  for  diabetics) 
should  be  encouraged.     If  changes  in  the  process  of  care  or  delivery  setting 
are  anticipated,   the  acceptance  of  such  changes  will  be   increased  if  informa- 
tion is  provided  to  patients  in  advance. 
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5.  On  an  experimental  basis,   quality  assurance  programs  should  include  con- 
sumer or  patient  boards  to  hear  patient  complaints  and  evaluate  their  validity. 
The  use  of  patient  questionnaires  in  assessing  quality  should  be  tested,  as 
well  as  the  effect  of  asking  patients  to  review  their  own  medical  records. 
Patient  expectations  upon  seeking  care  might  be  determined  and  used  as  the 
basis  for  providing  patient  education  and  instituting  treatment;  the  influence 
of  expectations  on  compliance  and  outcome  could  then  be  determined.     A  more 
direct  involvement  of  consumers  with  providers  in  assessing  the  quality  of 
care  should  be  tested;  each  group  may  learn  from  the  other. 

6.  Existing  laws  should  be  exploited  whenever  possible  to  further  health 
education.     Informed  consent  requirements,   for  instance,  provide  a  unique 
opportunity  to  educate  the  patient,  rather  than  simply  to  obtain  an  unthink- 
ing agreement   to  treatment. 

7.  The  consumer's  role  in  governance  and  policymaking  requires  careful 
documentation  and  analysis  to  facilitate  more  responsible,  comfortable,  and 
effective  relationships  with  health  care  professionals. 
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U.S.  Congress,  Senate,  Health  Maintenance  Organization  Act  of  1973,  Pub.  L. 
93-222,   93d  Cong.,  1st  sess.,   1973,   S.  14. 


Chapter  2 


BACKGROUND  AND  METHODS 


A  great  increase   in  federal  spending  for  medical  care  programs  over  the  past 
decade  has  more  recently  been  accompanied  by  congressional  concern  about  the 
cost  and  quality  of  that  care.     More  specific   interest  in  the  effectiveness 
of  methods  for  monitoring  the  quality  of  care  led  to  a  provision  in  the  Health 
Maintenance  Organization  Act  of  1973  (Public  Law  93-222),  which  called  for  a 
comprehensive  study  of  health  care  quality  assurance  programs.   [1]     The  study 
was  to  include  an  analysis  of  the  strengths,  weaknesses,  and  costs  of  quality 
assurance  prototypes;  identification  of  research  needs;  design  of  prospective 
methods  for  evaluating  quality  assurance  programs;  and  the  development  of 
specifications   for  an  effective  health  care  quality  assurance  system. 

Although  the  study  conducted  by  the  Institute  of  Medicine  and  reported  here 
stems  from  this  legislative  mandate,   it  is  more  limited  in  scope.     Its  general 
objective  is  "to  provide  an  explication  and  synthesis  of  current  knowledge 
about  the  study  and  development"  of  health  care  quality  assurance  systems, 
based  primarily  on  existing  documentation.    [2]     The  study  is  not  intended  to 
evaluate  Professional  Standards  Review  Organizations  (PSROs).   [3]     The  PSRO 
program  is  a  relatively  recent,   large-scale  undertaking  which  is  incomplete 
at  this  time.     Nevertheless,   some  quality  assurance  programs  reviewed  were 
PSROs,  and  some  recommendations  refer  specifically  to  the  PSRO  program. 


SPECIFIC  STUDY  OBJECTIVES 
Specific  study  objectives  were: 

•  To  describe  and  assess  the  impact  of  operational  quality  review  programs, 
based  on  existing  literature  and  supplemented  by  observations  and  data 
from  site  visits  ; 

•  To  review  seven  topics  (discussed  below)  designated  as  "priority  areas" 
because  of  their  importance  in  determining  the  effectiveness  of  quality 
assurance  programs  and  the  absence  of  reviews  to  integrate  and  analyze 
relevant  information; 

•  To  delineate  areas  in  which  additional  research  and  evaluation  are  re- 
quired; and 
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•  To  assess  the  reliability  of  hospital  utilization  information  generated 
by  private  abstracting  services  and  based  on  abstracts  of  the  hospital 
medical  record. 

Study  findings  related  to  the  first  three  objectives  are  included  in  this 
report.     Study  methods  are  discussed  below.     Descriptive  comparisons  of  qualit 
assurance  programs  and  an  analysis  of  what  is  known  about  their  effectiveness 
are  in  Chapters  3  and  4.     Reviews  of  the  priority  areas  are  summarized  in 
Chapter  5.     Research  recommendations  are  presented  at  the  end  of  each  chapter 
and  consolidated  in  Appendix  A.     Chapter  1  is  a  review  of  the  findings  and 
recommendations  of  the  steering  committee. 

The  fourth  objective  was  intended  to  determine  whether  hospital  utilization 
information  processed  by  private  abstracting  services  is  sufficiently  reliable 
to  be  used  as  baseline  information  for  assessing  the  impact  of  PSROs  over 
time.     This  activity  required  extensive  field  work  and  the  analysis  of  primary 
data.     Although  it  was  conducted  under  the  direction  of  the  steering  committee 
it  was  a  separate  activity  and  is  reported  in  a  separate  volume.  [4] 

CONCEPTS  OF  QUALITY 

To  limit  the  scope  of  the  study,  considerable  attention  was  devoted  to  clarify 
ing  the  concepts  of  "quality"  and  "systems"  for  quality  assessment.  Many 
definitions  of  quality  have  been  proposed,  ranging  from  statements  of  the 
ideal  to  minimal  standards.     Clearly,   the  definition  of  quality  is  multi- 
dimensional and  complex.     Quality  may  be  viewed  in  relation  to  a  total  system 
for  health  care  delivery,  components  of  the  system,  or  particular  attributes 
of  components.     The  type  of  care  that  an  individual  considers  ideal  may  be 
unrealistic  from  a  national  perspective.     The  criteria  and  information  base 
for  assessing  quality  and  the  range  of  options  for  responding  to  inappropriate 
or  unacceptable  care  may  vary,  depending  on  the  patient  or  health  care  pro- 
vider being  assessed  and  the  person  or  program  making  the  evaluation.     It  is 
unlikely  that  any  single  definition  or  approach  to  quality  assessment  can 
satisfy  the  many  needs  of  all  who  are  concerned  with  the  quality  of  care. 

Because  of  these  complexities,   the  steering  committee  adopted  a  pragmatic 
view  of  quality.     Components  of  quality  that  have  been  identified  and  empha- 
sized by  existing  programs  were  accepted  within  the  focus  of  this  study. 
However,  particular  attention  was  given  to  programs  with  the  stated  purpose 
of  improving  the  health  status  and  satisfaction  of  patients.   This  broad 
categorization  was  limited  further  by  a  more  detailed  specification  of  the 
kinds  of  systems  to  be  considered. 

Characteristics  of  an  ideal  quality  assurance  system  were  specified:  the 
existence  of  an  organizational  entity  created  for  assessing  quality,  the 
establishment  of  standards  or  criteria  against  which  quality  is  assessed, 
a  routine  system  for  gathering  information,   assurance  that  such  information 
is  based  on  the  total  population  or  representative  sample  of  patients  or 
potential  patients,  a  process  for  providing  the  results  of  review  to  patients, 
the  public,  providers,   and  sponsoring  organizations,   and  methods  for  institut- 
ing corrective  actions. 
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SELECTION  OF  PROGRAMS 

Within  the  broad  view  of  quality  assurance  systems,  programs  for  detailed  study 
were  selected  to  include  different  approaches  to  quality  assurance  which  might 
influence  program  effectiveness  and  be  applied  in  other  settings.  Similarities 
among  programs  usually  were  more  striking  than  differences.     Nevertheless,  some 
distinguishing  characteristics  were  important. 

One  important   factor  was  the  portion  of  the  medical  care  spectrum  emphasized 
in  the  review  process — the  identification  of  people  who  need  medical  care,  the 
diagnostic  and  treatment  process,   patient   follow-up,  and  outcome  assessment.  [5] 
Some  examples  of  each  were  identified  and  included   in  the  study. 

Within  programs  which  review  the  process  of  care,  variations  in  types  of  review 
usually  were  not  as  great  as  variations  in  the  organizational  auspices  and 
structures  through  which  review  is  conducted.     Therefore,   the  study  includes 
examples  of  programs  conducted  by  statewide  foundations  or  medical  societies, 
government  bodies  such  as  Medicaid  agencies,  hospitals  or  other  facilities, 
prepaid  health  plans,  and  solo  offices.     As  an  alternative  to  an  institutional 
or  programmatic  approach,  one  might  concentrate  on  the  quality  of  all  care  for 
an  individual  or  population  regardless  of  the  setting  in  which  care  is  provided. 
However,  very  few  systems  have  taken  this  approach;  and  for  that  reason  it  is 
not  emphasized  in  this  report. 

The  state  of  development  of  the  review  program  was  influential  in  the  selection 
process.     Programs  usually  were  excluded  unless  they  had  sufficient  operating 
experience  to  provide  some  information  over  a  period  of  time. 

Finally,   so  that  available  resources  would  be  used  most  efficiently,  programs 
were  selected  on  the  basis  of  their  ability  to  provide  information  about  the 
priority  areas  and  their  geographic  proximity  to  investigators  conducting 
relevant  research. 

Review  programs  for  detailed  analysis  were  selected  in  stages.     Initial  sug- 
gestions from  the  steering  committee  and  others  and  review  of  existing  material 
resulted  in  the  identification  of  about   fifty  programs  which  were  categorized 
according  to  the  organizational  groupings  mentioned  above.     Information  was 
obtained  to  reflect  the  following  program  characteristics:     the  types  of  review 
conducted  (pre-admission  or  admission  certification,  concurrent  or  retrospec- 
tive review,  profile  analysis,  medical  care  evaluations,  outcome  assessments); 
type  of  corrective  actions  taken  (continuing  education,   incentives,  sanctions, 
other);  patient  coverage  (Medicare,  Medicaid,  other  than  federally  reimbursed); 
age  of  review  system;  geographic  location  and  degree  of  urbanization;  extent 
of  consumer  involvement;  and  presence  or  absence  of  cost  data. 

Illustrative  programs  were  selected  from  each  category  on  the  basis  of  this 
preliminary  information.     They  were  described  in  more  detail  and  later  visited 
by  field  team  members.     These  programs  are  listed  in  Appendix  B. 
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DATA  COLLECTION 

So  that  quality  assurance  programs  could  be  compared,  an  attempt  was  made  to 
obtain  similar  information  about  each  program,   including  a  description  of  the 
formal  characteristics  of  the  program,  an  evaluation  of  actual  or  expected 
impact,   and  information  about  unique  characteristics  which  might  influence 
program  effectiveness.     Because  substantiating  documentation  was  often  lacking, 
this   information  reflects  a  mixture  of  documented  and  undocumented  data  provided 
by  program  administrators  and  a  subjective  assessment  by  the  study  staff  and 
steering  committee.     An  outline  of  descriptors  was  developed  (see  Appendix  C) 
to  organize  the  information  and  indicate  whether  it  reflects  data  provided  by 
program  staff  or  a  judgment  by  persons  involved  in  the  Institute  of  Medicine 
study. 

Information  provided  by  program  staff  was  divided  into  five  categories: 
origins  and  description  of  review  system;  structure  of  the  system;  process 
of  review;  results;  and  long-range  implications.     Because  of  the  difficulty 
of  assessing  the  results  of  review,   information  was  gathered  about  a  wide 
range  of  items  including  methods  to  assess  the  attainment  of  goals,  impressions 
and  anecdotes  about  changes  resulting  from  review  activities,  and  changes  in 
aggregate  utilization  and  other  statistics  based  on  systematically  gathered 
data.     The   final  category,   long-range  implications,  was   intended  to  elicit 
opinions   from  the  staff  about  the  program's  flexibility  and  potential  for 
change  necessitated  by  increased  scientific  knowledge,  new  developments  in 
medical  practice,  or  social  and  economic   factors.     Questions  were  also  asked 
about  any  underlying  problems  (either  general  or  specific   to  the  setting) 
which  should  be  considered  in  developing  policy  and  research  recommendations. 

Assessment  by  the  Institute  of  Medicine  staff  and  steering  committee  included 
the  current  status  and  accomplishments  to  date  of  each  program  reviewed,  unique 
characteristics  which  may  influence  the  program's  effectiveness,  and  unresolved 
administrative,  methodological,  or  policy  issues.     The  notation  of  unresolved 
issues  was  not   intended  for  evaluative  purposes.     Instead,   the  objective  was  to 
determine  whether  common  problems  existed  across  systems  and,   if  so,  to  incorpo- 
rate them  in  study  recommendations.     Required  follow-up  activities  were  also 
noted,  as  well  as  unanticipated  factors  which  may  have  influenced  staff  ability 
to  obtain  the  required  information. 

Before  the  site  visit  the  outline  of  descriptors  was  completed  as  fully  as 
possible  based  on  information  from  the  program  administrators,  other  studies, 
and  telephone  conversations.     The  site  visit  was  then  devoted  primarily  to 
observing  the  manner  in  which  the  system  actually  functions,  as  distinguished 
from  the  formal  written  description,  and  reviewing  the  motivations   for  initiating 
the  program,  reasons   for  selecting  the  particular  approach  to  review,  incentives 
built  into  the  program  to  facilitate  behavior  change,  perceptions  of  participants 
about  success  to  date,   impressions  of  underlying  problems,   areas  requiring 
further  development,  and  any  available  data  suggesting  impact. 

The  outline  of  descriptors  was  not  applied  as  a  formal  questionnaire  or  survey 
instrument,  but  rather,  constituted  a  general  guide  to  the  kinds  of  information 
required,  which  varied,  depending  on  the  program.     For  some  programs,  specific 
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information  items  were  not  relevant  and  were  omitted;   in  other  cases  desired 
information  could  not  be  obtained. 

Most  program  administrators  were  extremely  cooperative  and  willing  to  provide 
extensive  descriptive  material.     However,   some   information  did  not  arrive 
early  enough  to  be  reviewed  before  the  site  visit.     The  visit  was  then  con- 
ducted on  the  basis  of  information  from  other  sources.     A  few  program  admin- 
istrators were  reluctant  to  provide  written  information  in  advance,  so  the 
material  reflects  what  was  learned  during  a  relatively  short  period  of  time 
on  site,  supplemented  by  whatever  material  was  provided  later.  Occasionally, 
program  officials  were  unwilling  to  provide  specifically  requested  information, 
especially  about  costs  and  evidence  of  impact. 

The  Institute  of  Medicine  usually  was  represented  during  the  site  visits  by 
three  or  four  people,   including  a  member  of  the  steering  committee  or  a  consul- 
tant or  both,  as  well  as  full-time  study  staff.     Whenever  possible,  a  physi- 
cian was  included.     Representatives  from  the  program  under  review  usually  in- 
cluded policymaking  and  administrative  officials,  as  well  as  people  involved 
in  the  day-to-day  review  and  data  gathering  activities.     Most  visits  were 
conducted  in  one  day,   although  the  length  of  time  on  site  ranged  from  four 
hours  to  two  days. 

PRIORITY  AREAS 

Seven  topics  were  designated  priority  areas  because  of  their  importance  in 
determining  the  effectiveness  of  quality  assurance  programs  and  the  absence 
of  reviews  to  integrate  and  analyze  pertinent  information  and  delineate  gaps 
in  knowledge.     The  areas,  which  are   interrelated,   are  outcome-oriented  ap- 
proaches to  quality  assurance;  quality  assurance  for  ambulatory  care;  quality 
assurance  for  long-term  care;  assessment  of  costs  and  cost  effectiveness; 
medical  care  evaluation  studies;  methods  for  changing  behavior  patterns  of 
health  care  providers;  and  patient  and  consumer  involvement   in  quality  assur- 
ance programs.     Because  medical  care  evaluation  studies  and  cost  effective- 
ness issues  are  integral  parts  of  the  review  of  operational  programs,  the 
findings  and  recommendations  in  these  areas  are  included  in  Chapters  3  and  4. 
The  five  remaining  priority  areas  are  summarized  in  Chapter  5;   future  priority 
area  reports  will  be  published  as  supplements  to  this  volume. 

In  exploring  each  area  the  basic  approach  was  to  review  existing  literature 
and  current  research  in  an  attempt  to  delineate  what  is  currently  known,  the 
adequacy  of  that   information  for  generating  policy  recommendations,  and  speci- 
fic areas  in  which  further  research  and  development  are  needed.     The  relation- 
ship of  current  research  to  operational  quality  assurance  programs  was  noted. 

Institute  of  Medicine  staff  was  responsible  for  initial  drafts  of  background 
papers,  which  were  reviewed  by  steering  committee  members  and  consultants  and 
formed  the  basis  for  recommendations. 
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LIMITATIONS 


The  adequacy  of  the  priority  papers  rests  primarily  on  the  care  with  which  the 
literature  was  analyzed  and  the  soundness  of  judgment  on  which  the  conclusions 
are  based.     The  papers  are  essentially  literature  reviews  rather  than  policy 
statements,  and  generally  provide  a  guide  for  future  activities  rather  than 
immediate  decisions.     The  implications  of  the  assessment  of  quality  assurance 
systems  may  be   felt  more  immediately,  however.     Thus,   the  limitations  should 
be  clearly  specified. 


The  non-systematic  nature  of  selecting  quality  assurance  programs  for  review 
limits  the  generalizability  of  the  information  reported.     The  programs  visited 
do  not  constitute  a  representative  national  sample  and  the  conclusions  do  not 
reflect  the  status  of  quality  assurance  systems  across  the  country.     One  hears 
reports  of  perfunctory  implementation  of  the  PSRO  program.     Such  programs  were 
not  visited,  however,  because  the  purpose  of  the  study  was  to  learn  about  re- 
view mechanisms  that  might  be  instituted  elsewhere.     Similarly,  some  institu- 
tions are  reported  to  have  serious  deficiencies  in  the  quality  of  care,  which 
are  readily  apparent  and  do  not  require  a  quality  review  system  for  their  de- 
tection.    These  facilities  also  were  excluded  from  the  study. 

The  steering  committee  believed  that  by  concentrating  on  what  have  been  re- 
garded as  some  of  the  "better"  quality  assurance  programs,   it  would  be  able  to 
ascertain  the  effects  of  such  programs  under  ideal  circumstances.     If  they  were 
found  to  be  generally  effective,  one  might  assume  that  similar  success  could  be 
attained  elsewhere  by  using  existing  programs  as  models.     On  the  other  hand,  if 
serious  questions  were  raised,  one  might  assume  that  more  severe  reservations 
might  be  warranted  elsewhere. 

The  inadequacy  of  documentation  and  limited  time  on  site  must  be  viewed  as  lim- 
itations of  the  data.     Whenever  programs  are  identified  by  name,  the  material 
has  been  reviewed  by  program  officials  to  ensure  the  accuracy  of  factual  infor- 
mation.    The  interpretation  and  recommendations  are  the  responsibility  of  the 
steering  committee.     A  conscious  attempt  has  been  made  to  provide  a  balanced 
presentation,  which  can  constitute  a  framework,  for  conducting  a  more  detailed 
and  definitive  assessment  of  health  care  quality  assurance  programs  in  the 
future . 
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The  distinction  between  process  and  outcomes-oriented  reviews  refers  to 
Donabedian's  conceptualization  of  approaches  to  quality  assessment.     He  defines 
three  approaches  as  follows:  1)  structural — "the  evaluation  of  the  settings 
and  instrumentalities  available  and  used  for  the  provision  of  care" ...  including 
the  available  resources  and  manner  in  which  they  are  organized;   2)  process- 
oriented — "the  evaluation  of  the  activities  of  physicians  and  other  health 
professionals  in  the  management  of  patients" — usually  in  relationship  to 
implicit  or  explicit  professional  standards;  and  3)  outcome-oriented — "the 
evaluation  of  end  results  in  terms  of  health  and  satisfaction."  American 
Public  Health  Association,  A  Guide  to  Medical  Care  Administration,  2  vols. 
(New  York):     American  Public  Health  Association,   1969),  vol.  2:     Medical  Care 
Appraisal-Quality  and  Utilization,  by  Avedis  Donabedian,  pp.  2-3. 


Chapter  3 


QUALITY  ASSURANCE  PROGRAMS 


The  quality  assurance  programs  reviewed  in  detail  are  described  in  this 
chapter.     Because  there  are  many  similarities  among  programs,  much  of  the 
discussion  is  general.     Specific  programs  are  discussed  separately  only 
if  unique  characteristics  deserve  more  detailed  consideration.     A  tabular 
comparison  of  programs  on  most  characteristics  discussed  below  is  included 
as  Chart  1. 


GOALS  AND  ORIENTATIONS 

The  explicit  goals  of  most  programs  relate  to  the  necessity  of  ensuring  high 
quality  medical  care  at  reasonable  cost.     Some  statements  of  objectives  also 
incorporate  the  effective  and  efficient  utilization  of  services.     One  group 
referred  to  efficient  statewide  distribution  of  services  (Colorado);  another, 
to  the  continuity  of  care  (Indian  Health  Service).     In  general,  goals  are  not 
defined  in  specific  terms.     It  is  possible,  however,   to  infer  more  detailed 
objectives  and  priorities  by  examining  the  orientation  of  the  program's  review 
process  and  the  way  in  which  it  functions. 

The  emphasis  given  to  reviewing  the  cost  rather  than  quality  of  care  is  dif- 
ficult to  determine  conclusively,  because  the  two  are  intertwined.     In  the 
case  of  hospital  and  PSRO  programs,   the  amount  of  time  and  money  devoted  to 
concurrent  review  activities,  which  deal  primarily  with  utilization  patterns, 
could  imply  a  primary  emphasis  on  cost.     But  eliminating  exposure  to  the  risk 
of  unnecessary  medical  intervention  and  potential  iatrogenic  illness  through 
control  of  utilization  has  implications  for  quality.     Although  most  program 
administrators  place  more  emphasis  on  concurrent  review  than  on  medical  care 
evaluation  and  the  quality  of  medical  practice,  many  express  the  desire  to 
concentrate  on  quality  in  the  future.     A  precise  separation  of  the  effect  of 
quality  assurance  programs  on  cost  from  their  impact  on  quality  of  care  may 
be  impossible.     It  may  be  more  appropriate  to  refer  to  cost  components  and  non 
cost  components  of  quality  review.     A  subjective  determination  of  the  rela- 
tive importance  of  those  two  components  for  any  given  program  can  be  made  by 
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CHART  1.     CHARACTERISTICS  OF  QUALITY  ASSURANCE  PROGRAMS  VISITED 


Bethesda  Lutheran 

Hospital 

St.  Paul,  Minn. 


Mt .  Sinai  Hospital 
New  York  City 


Overlook  Hospital 
Summit,  New  Jersey 


DATE  QUALITY 
ASSURANCE 
ACTIVITIES 
INITIATED 


Audit  1967 
Utilization  Review 
1971 


Initial  program  1973 
Expanded  program  1975 


Early  1960s 


GEOGRAPHIC 
&  PATIENT 
COVERAGE 


All  hospitalized 
patients 


All  hospitalized 
patients 


All  hospitalized 
patients 


TYPES  OF  Admission  certification, 

REVIEW  continued  stay  review, 

MCEs,  patient  care 
monitor  (profiles) 


Pre-admission  screening 
(non-emergency) , 
admission  certification, 
continued  stay  review, 
MCEs 


Limited  concurrent 
review,  MCEs  (special 
studies ) 


HOSPITAL 

DELEGATION 

STATUS 


Delegated 


Delegated 


Not  applicable 
(no  local  PSRO) 


CORRECTIVE 

ACTION 

EMPHASIS 


SOURCE  OF 
COST  DATA 


Primarily  educational; 
full  review  and  re- 
stricted privileges 
poss  ible 

PSRO  reports;  hospital 


Peer  pressure  and 
education 


Hospital  budget 


Peer  pressure  and 
education 


Hospital  budget 


AVAILABLE  Reported  quality  improve- 

IMPACT  ments;   length  of  stay 

DATA  data 


Recommended  quality  im- 
provements;  length  of 
stay  data 


Reported  quality  im- 
provements; length 
of  stay  data 


FACTORS  Strong  leadership  and 

INFLUENCING  administrative  changes; 
EFFECTIVENESS     need  for  improvement; 

support  of  board  of 

d  irectors 


Demand  for  beds  despite 
excess  in  the  area 


Strong  leadership; 
shortage  of  beds 
in  area 


SPECIAL  Patient  care  monitor; 

FEATURES  experience  with  MCEs; 

Increased  responsibility 
of  nurse  coordinator 


Pre-admission  ap- 
proval of  treatment ; 
integrated  discharge 
planning 


Selective  concur- 
rent review; 
experience  with 
MCEs 


[a  J  Current  or  prior  EMCRO  support.     lb  J  Conditional  PSRO.     [c] Research-oriented . 

[d]  This  program  was  not  visited;  description  based  on  Brook  and  Williams,  "Evaluation 

of  the  New  Mexico  Peer  Review  System  1971-1973"  Medical  Care,  forthcoming. 
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Colorado  Foundation  for 
Medical  Care 
Denver  [a,b] 


Foundation  for  Health 
Care  Evaluation 
Minneapolis  [b] 


Medical  Care  Foundation 
of  Sacramento 
Sacramento  [a,b] 


DATE  QUALITY 
ASSURANCE 
ACTIVITIES 
INITIATED 

GEOGRAPHIC 
&  PATIENT 
COVERAGE 


Foundation  created  1970 
Ambulatory  review  1972 
Hospital  review  1973 

Statewide 

Hospital — Medicare , 
Medicaid  and 
commercial;  ambulatory — 
commerc  ial 


Fee  review  1969 
Hospital  review  (PSRO) 
June  1974 


7  counties 

All  hospitalized  patients 


Foundation  created 
1958 

Hospital  review 

5  counties 
Hospital — Medicare , 
Medicaid,  commercial; 
ambulatory — Medicaid 
and  commercial 


TYPES  OF  Hospital:  admission  certi- 

REVIEW  fication,  continued  stay, 

profiles,  MCEs; 

Ambulatory:  claims 


Admission  certification; 
continued  stay  review; 
MCEs;  patient  care 
monitor  (profiles) 


Hospital:  pre-admis- 
sion certification  and 
continued  stay; 
Long-term  care; 
Ambulatory:  claims 
and  profiles 


HOSPITAL 

DELEGATION  Non-delegated  at  time  of        21  of  37  hospitals  Non-delegated 

STATUS  visit;  now  mixed  delegated 


CORRECTIVE  Hospital:  peer  pressure 

ACTION  and  denials 

EMPHASIS  Ambulatory:   fee  reduction 


Education  and  peer 
pressure 


Hospital:  denial  of 
admission 

Ambulatory:  denial  of 
payment 


SOURCE  OF  PSRO  reports  and  special 

COST  DATA  tabulations 


PSRO  reports  availa- 
ble but  not  analyzed 


PSRO  reports  availa- 
ble but  not  analyzed 


AVAILABLE  Length  of  stay  and  bed 

IMPACT  days  saved;  cost  avoid- 

DATA  ance 


Anecdotal 


Utilization  data; 
quality  (anecdotal); 
fee  adjustments 


FACTORS 

INFLUENCING 

EFFECTIVENESS 


Strong  leadership; 
centralized  management 
of  regionalized  organiza- 
tional structure 


Consumer  involvement; 
prior  experience  in 
claims  adjudication  and 
fee  modifications 


Strong  leadership 


SPECIAL 
FEATURES 


Regionalized  review  using 
statewide  standards 


Potential  review  of 
hospitalized  popula- 
tion 


Pre-admiss  ion 
certification 
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CHART  1.     CHARACTERISTICS  OF  QUALITY  ASSURANCE  PROGRAMS  VISITED  (continued) 


Multnomah  Foundation  for 
Medical  Care 
Portland,  Oregon  [a,b] 


National  Capital 
Medical  Foundation 
Washington,  D.C.  [b] 


New  Mexico  Foundation 
for  Medical  Care 
Albuquerque,  [a,d] 


DATE  QUALITY 
ASSURANCE 
ACTIVITIES 
INITIATED 


Foundation  created  1961 
Hospital  review  1972 


Foundation  created  1973 
Hospital  review  Sept. 
1975 


1971 


GEOGRAPHIC  County-wide 

&  PATIENT  Hospital:  Medicare,  Medi- 

COVERAGE  caid;  commercial 


TYPES  OF  Hospital:  admission  certi- 

REVIEW  fication,  continued  stay, 

MCEs,  profiles; 

Long-term  care; 

Amb:  developmental 


HOSPITAL 

DELEGATION 

STATUS 


CORRECTIVE 

ACTION 

EMPHASIS 


5  fully  delegated,  8 
partially  delegated 
and  1  non-delegated 
hospitals 

Education  and  peer 
pressure 


Washington,  D.C. 
Medicare,  Medicaid 


Admission  certification, 
continued  stay  review, 
MCEs, 

profiles  (preliminary) 


3  fully  delegated,  5 
partially  delegated 
and  5  non-delegated 
hospitals 

Education  and  peer 
pressure;  denials 
of  payments 


Statewide 
Medicaid 


Integrated  claims  re- 
view for  ambulatory, 
hospital,  long-term 
care,   labs,  X-ray,  in- 
jections ; 

Hospital:  admission 
certification  and  con- 
tinued stay  review; 
LTC:   level  of  care 

Not  applicable 


Denial  of  payment 
and  education 


SOURCE  OF 
COST  DATA 


AVAILABLE 

IMPACT 

DATA 


FACTORS 

INFLUENCING 

EFFECTIVENESS 


SPECIAL 
FEATURES 


PSRO  reports  and 
letter  from 
foundation 

Length  of  stay  data; 
quality  (anecdotal) 


Prior  experience 


Problem  processing 
system  and  ambulatory 
data  system  (potential 
only) 


PRSO  reports  available 
but  not  analyzed 


Not  available 
(review  only  recently 
initiated) 


Not  yet  determined 


Secondary  source 


Payment  denials  by 
type  of  claim  and 
reason;  quality 
changes  for  se- 
lected procedures 

Cooperation  between 
state  and  founda- 
tion;  strong  tech- 
nical and  computer 
support 

Integrated,  compu 
terized  review  of 
comprehensive  range 
of  services 


[a]  Current  or  prior  EMCRO  support.     lb  J  Conditional  PSRO.     [c]  Research-oriented . 
[d]  This  program  was  not  visited;  description  based  on  Brook  and  Williams,  "Evaluation 
of  the  New  Mexico  Peer  Review  System  1971-1973"  Medical  Care,  forthcoming. 
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San  Joaquin  Foundation 
for  Medical  Care 
Stockton,  California 


San  Joaquin  PSRO 
Stockton , 
California  [b] 


Utah  Professional 

Organization 

Salt  Lake  City  [a,b] 


DATE  QUALITY 
ASSURANCE 
ACTIVITIES 
INITIATED 

GEOGRAPHIC 
&  PATIENT 
COVERAGE 


Foundation  created  1954         Predecessor  1968 
Manual  review  1960  Hospital  review  1974 

Patterns  of  Treatment  1970    Long-term  care  1975 


4  counties 

Commercial,  including 
Medicare 


TYPES  OF  Ambulatory  claims  review 

REVIEW  and  profile  analysis 


5  counties 
Commercial  and 
Medicare 


Hospital  and  long 
term  care:  admission 
certification,  con- 
stay  review,  MCEs 


UPRO  formed  July  1971 
Hospital  review  1971 
Ambulatory  review  1972 

Statewide 

Hospital:  Medicare, 
Medicaid,  commer- 
cial ; 

Ambulatory:  Medicaid 

Hospital:  admission 
certification,  con- 
tinued stay  review, 
MCEs,  profiles; 
Amb:  claims  review  and 
provider  and  patient 
profiles 


HOSPITAL 

DELEGATION 

STATUS 


Not  applicable 


9  partially  dele- 
gated 

4  non-delegated 
hospitals 


Non-delegated 


CORRECTIVE 

ACTION 

EMPHASIS 


Denial  of  payment 
and  total  review 


Primarily  left  to 
discretion  of 
hospital;  poten- 
tial total  review 


Primarily  educa- 
tional; non- 
certification  of 
ambulatory  claims 


SOURCE  OF  Foundation;  secondary 

COST  DATA  sources 


PSRO  reports  available 
but  not  analyzed 


PSRO  reports  and 
communication  with 
foundation 


AVAILABLE  Fee  adjustments; 

IMPACT  changes  in  utiliza- 

DATA  tion  (anecdotal); 

quality  (anecdotal) 


Length  of  stay  data; 
quality  (anecdotal) 


Some  length  of  stay  da- 
ta; quality  (anecdot- 
al); certification  de- 
nial ;  secondary  sources 


FACTORS 

INFLUENCING 

EFFECTIVENESS 


Strong  leadership; 
lengthy  experience; 
discontinuation  of 
Patterns  of  Treatment 


Earlier  San 
Joaquin  Foundation 
activities 


Strong  leadership; 
homogeneous  setting; 
careful  develop- 
ment phase 


SPECIAL  Patterns  of 

FEATURES  Treatment  program 


Experimental  approach 
to  MCEs;  PACE  ambula- 
tory review  program; in- 
creased responsibility 
of  nurse  coordinator 
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CHART  1.     CHARACTERISTICS  OF  QUALITY  ASSURANCE  PROGRAMS  VISITED  (continued) 


Texas  Department  of 
Public  Welfare,  Austin 


Indian  Health  Service 
Health  Information 
System,  Tucson  [a] 


Columbia  Medical  Plan- 
Johns  Hopkins  Health 
Services  Research 
Center 

Columbia^  Md.  [a,c] 


DATA  QUALITY 
ASSURANCE 
ACTIVITIES 
INITIATED 


Elements  started  by 
1971 


Health  Information 
System  1969;  evaluation 
program  1974 


Research  project 
funded  1973 


GEOGRAPHIC 
&  PATIENT 
COVERAGE 


Statewide 
Medicaid 


Sells  service  unit  of 
Indian  Health  Service 
Papago  tribe 


Columbia,  Md. 
Health  Plan 
enrollees 


TYPES  OF  State  monitor  of  utiliza- 

REVIEW  tion  profiles  (hospital 

and  ambulatory);  concur- 
rent review  of  long-term 
care;   fiscal  intermedi- 
aries at  risk  and  reponsi- 
ble  current  hospital  review 


Ambulatory:  concur- 
rent and  retrospec- 
tive using  automated 
information  system 


Patient  perceptions 
of  access  and  out- 
comes linked  with 
record  review 


HOSPITAL 

DELEGATION 

STATUS 


Not  applicable 


Not  applicable 


Not  applicable 


CORRECTIVE 

ACTION 

EMPHASIS 


Denial  of  payment; 
education  of  provider 
and  patient 


Neglible 


Feedback  of  review 
results  to  providers 


SOURCE  OF 
COST  DATA 


Not  available 


Program  staff 


Program  staff 


AVAILABLE 

IMPACT 

DATA 


FACTORS 

INFLUENCING 

EFFECTIVENESS 


SPECIAL 
FEATURES 


Quality  (anecdotal); 
length  of  stay  data; 
expenditure  data 


Strong  leadership; 
governmental  authority; 
integration  of  hospital 
and  ambulatory  data 


Comprehensive  data  system; 
recipient  education  pro- 
gram; medical  assistance 
record  book 


Quality  (anecdotal) 


Self-contained  de- 
livery system; 
consumers  involved 
in  policymaking  and 
providing  health 
services 

Health  information 
system  transf errable 
to  organization  with 
defined  population 


Quality  (anecdotal); 
research  findings  from 
application  of  problem 
status  measure 

Leadership  and  support 
of  parent  organization 
(Johns  Hopkins  Health 
Services  Research  & 
Development  Center) 

Pat ient -reported 
problem  status 
measure 


[aj  Current  or  prior  EMCRO  support.     lb  J  Conditional  PSRO.     [c\  Research-oriented . 

[d]  This  program  was  not  visited;  description  based  on  Brook  and  Williams,  "Evaluation 

of  the  New  Mexico  Peer  Review  System  1971-1973"  Medical  Care,  forthcoming. 
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Kaiser-Permanente  Medical 
Group 

Los  Angeles  [a,c] 


Kaiser-Permanente 
Medical  Group 
Oakland,  California 


UCLA-EMCRO 

Los  Angeles  [a,c] 


DATE  QUALITY 
ASSURANCE 
ACTIVITIES 
INITIATED 


Research  project 
funded  1973 


1969 


Research  project 
funded  1973 


GEOGRAPHIC 
&  PATIENT 
COVERAGE 


Sample  of  prepaid 
health  plan  enrollees 
with  6  diagnoses 


Prepaid  health  plan 
enrollees 


Patients  with 
selected  diagnoses 
or  conditions 


TYPES  OF  Experimenting  with 

REVIEW  imput,  process,  outcome 

and  access  measures; 

emphasis  on  outcomes 


CQAS  Program- 
primarily  retro- 
spective review  of 
problems  in  ambula- 
tory and  hospital 
care 


Treatment  algorithms 
(MAPs)  suitable  for 
MCEs  and  potentially 
for  concurrent 
review 


HOSPITAL  Not  applicable  Delegation  for  MCEs  Not  applicable 

DELEGATION 

STATUS 


CORRECTIVE  Not  yet  developed  Education  and  admin-  Feedback  of  review 

ACTION  istrative  changes  findings  to  pro- 

EMPHASIS  viders 


SOURCE  OF  Included  in  research 

COST  DATA  design;  not  yet 

available 


Personal  communication 


Personal  communication 


AVAILABLE 

IMPACT 

DATA 


Not  yet  available 


Quality  improvements 
and  administrative 
changes  (anecdotal) 


Preliminary  results 
from  applying  MAPs 


FACTORS 

INFLUENCING 

EFFECTIVENESS 


SPECIAL 
FEATURES 


Self-contained  organiza- 
tional structure  and 
leverage  of  prepaid 
group  health  plan 


Attempt  to  develop 
proxy  measures  of 
outcome 


Strong  conceptual 
leadership;  self- 
contained  structure 
and  leverage  of  pre- 
paid group  health 
plan 

CQAS  program 


Strong  conceptual 
leadership;  pre- 
vious research 
experience ; 
academic  environment 


Criteria  mapping 
system  (MAPs) 
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referring  to  Chart  2.     On  this  basis,   perhaps  eight  of  the  18  programs  visited 
are  primarily  interested  in  quality;  and  five,   in  cost.     Five  programs  could 
not  be  categorized. 


CHART  2.     COST  AND  NON-COST  PROGRAM  CHARACTERISTICS 


Characteristics 

Cost  Emphasis 

Non-Cost  Emphasis 

Emphasis  of  concurrent 
review* 

Certifying  need  for 
admission  and  con- 
tinued stay  only  at 
specific  points  in 
t  ime 

Monitoring  the 
content  of  care 
throughout  the 
hospitalization 

Interest  and  resources 
devoted  to  medical  care 
evaluat  ions* 

Minimal 

Considerable 

Emphasis  of  ambulatory 
care  review 

Attention  only  to 
single  visit  and 
reasonableness  of 
services  as  re- 
flected by  relative 
value  scale 

Attention  to 
content  and 
outcome  of  care 
provided  over  a 
period  of  time 

Emphasis  of  corrective 

Simple  withdrawal  of 

LcLL  11  ILaL  lull    UI  UtrlllclJ- 

of  payment  without  in- 
formation feedback. 

Mechanism  to  tell 

rn-  n  w  l  A    r"    wh \j    h  l  Q 
pi uv luci     wny    n id 

judgment  was 
questioned  and  to 
design  educational 
and  other  correct- 
ive programs  to 
improve  future 
quality  of  care 

Examples  of  effective- 
ness provided  by  program 
administrators 

Reduction  in  days  of 
hospital  care  or  ambu- 
latory claims 

Specific  instances 
in  which  the 
quality  of  care 
has  been  improved 

*These  types  of  review  are  discussed  below. 


Most  programs  are  concerned  primarily  with  users  of  institutional  or  ambula- 
tory health  services,  rather  than  people  who  do  not  use  services;  with  few 
exceptions  they  do  not  question  access  to  care  or  under-ut i 1 izat ion  of  ser- 
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vices.     Review  processes  focus  on  care  provided  in  a  particular  facility  or 
group  of  facilities  or  physicians'  offices  within  a  geographic  or  PSRO  region. 
The  patients  whose  care  is  reviewed  usually  are  beneficiaries  of  Medicare  or 
Medicaid  programs  or  insured  groups  whose  insurance  companies  have  contracts 
for  review.     The  potential  benefits  from  review  may  not  be  available  to  the 
general  public.     Furthermore,  available  population-based  data  cannot  be  used 
to  assess  the  effect  of  review  on  health  status  because  of  selective  patient 
coverage . 

Among  the  exceptions  is  the  health  care  evaluation  component  of  the  Health 
Information  System  (HIS)  developed  by  the  Indian  Health  Service  (IHS)  in 
Tucson.     HIS  retrieves  information  on  all  care  provided  to  tribal  members, 
including  care  provided  in  homes  and  schools.     Except  for  those  who  have 
never  received  health  care  from  the  IHS  (about  five  percent  of  the  popula- 
tion),  follow-up  care  and  under-utilization  can  be  monitored.     Thus,  HIS 
covers  almost  all  care  provided  to  almost  a  total  population. 

The  prepaid  group  health  plans  included  in  this  study  could  monitor  the  qual- 
ity of  care  provided  to  the  total  enrolled  population,  but  primary  emphasis  is 
on  users,  rather  than  non-users.     The  experimental  project  conducted  by  the 
Johns  Hopkins  University  Health  Services  Research  Center  and  the  Columbia, 
Maryland  Health  Plan,  however,  has  surveyed  enrollees  who  have  not  used  its 
services  to  determine  their  health  needs  and  reasons  for  not  using  the  plan. 

The  Foundation  for  Health  Care  Evaluation  (a  PSRO  located  in  the  Minneapolis- 
St .  Paul  area)  has  the  potential  for  reviewing  the  quality  of  hospital  care 
given  to  a  total  population,  since  staff  in  participating  hospitals  are  en- 
couraged to  review  care  rendered  to  all  patients.     Total  review  has  been  at- 
tained to  date,  but  some  hospitals  have  yet  to  be  encompassed  within  the  PSRO 
umbrella . 

Some  ambulatory  claims  review  programs  are  developing  the  capacity  to  address 
under-utilization  in  a  limited  way.     This  is  discussed  in  the  section  on  ambu- 
latory review. 


SPONSORSHIP,  MOTIVATION,  AND  LEADERSHIP 

Except   for  prepaid  health  plans  and  the  Indian  Health  Service,  most  programs 
are  sponsored  directly  by  medical  societies  or  by  foundations  created  under  th 
leadership  of  a  medical  society.     Responsibility  for  hospital-based  programs  o 
fically  resides  with  the  boards  of  directors;  however,  medical  staff  leadershi 
was  instrumental   in  program  development  in  each  case.     Universities  and  major 
medical  centers  seldom  initiated  review  programs.     In  no  instance  did  an  organ 
ized  consumer  group  provide  the  impetus  to  establish  a  quality  assurance  pro- 
gram. 

Motivations  for  creation  of  programs  were  mixed.     Most   frequently  mentioned 
reasons  were  an  expressed  need  on  the  part  of  the  medical  profession  to  dem- 
onstrate public  accountability  and  the  desire  to  obviate  further  regulation 
by  government  agencies  and  third  party  carriers.     Only  one  program  was  started 
because  it  was  known  that  care  was  poor. 
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An  inability  to  estimate  precisely  the  margin  by  which  care  could  be  im- 
proved was  strikingly  apparent  in  all  programs  visited.     Review  programs 
may  provide  a  lever  for  closely  scrutinizing  physicians  who  give  less  than 
optimal  care.     (Estimates  of  physicians  with  severe  deficiencies  are  seldom 
more  than  five  percent  of  the  total  physician  population  and  never  adequately 
documented  with  criteria.)     But  nowhere  was  a  program  official  able  to  offer 
even  a  rough  estimate  of  the  extent  to  which  medical  care  should  be  improved 
in  its  totality.     This  point  is  significant,  since  without  a  measure  of  the 
amount  of  deficient  or  inappropriate  care,   it  becomes  difficult  to  determine 
whether  a  program  is  achieving  its  objectives  and  whether  the  resultant  im- 
provement is  sufficient   to  justify  the  expenditures  required  to  maintain  the 
program. 

In  almost  every  program  it  was  possible  to  identify  an  individual  who  was 
instrumental  in  generating  support  for  the  creation  of  the  review  program 
and  providing  strong  leadership  from  the  development  of  the  program  into  its 
operational  phase.     However,   it  is  important  to  note  that  most  programs  also 
have  a  full-time  professional  staff  to  oversee  day-to-day  activities  and  main- 
tain program  continuity.     In  a  few  programs,   the  leadership  role  has  passed 
from  the  initial  person  to  a  successor,  thus  demonstrating  the  viability  of 
the  program.     Furthermore,  the  establishment  of  a  successful  program  can  stim- 
ulate the  creation  of  similar  programs  throughout  a  geographic  area.     For  ex- 
ample, the  quality  assurance  program  at  Bethesda-Lutheran  Hospital  has  been 
used  as  one  of  the  prototypes  in  the  creation  of  the  PSRO  of  the  Foundation 
for  Health  Care  Evaluation.     Some  of  the  key  individuals  were  the  same  in  both 
programs.     The  experience  of  the  hospital  program  was  packaged  into  a  review 
manual  for  use  in  other  area  hospitals.     The  Utah  Professional  Review  Organiza- 
tion (UPRO)  has  evolved  by  expanding  its  activities  as  it  acquired  experience 
and  capability.     It  grew  from  an  experimental  program  in  a  few  hospitals  with 
selected  categories  of  patients  into  a  larger  program  which  gradually  accepted 
review  responsibility  for  additional  third  party  carriers.     Now,  UPRO  covers 
almost  all  hospitals  in  the  state. 


LINKS  WITH  EXTERNAL  GROUPS  OR  AGENCIES 

By  necessity,   staffs  of  almost  every  program  have  developed  working  relation- 
ships with  third  party  carriers,   state  Medicaid  agencies,   and  Medicare  fiscal 
intermediaries  for  which  the  program  provides  review.     Non-physician  health 
professionals,  hospital,  and  occasionally  consumer  groups  may  have  an  official 
advisory  responsibility.     Beyond  this,  however,  working  relationships  with  other 
components  of  the  health  care  delivery  system  are  minimal.     Links   to  planning 
and  regulatory  bodies  are  tenuous  and  perhaps  antagonistic. 

There  is  little  consumer  or  public   involvement  at  the  policymaking  level,  with 
two  exceptions.     The  Foundation  for  Health  Care  Evaluation  includes  consumers 
on  its  board  of  directors  and  also  has  a  consumer  liaison  advisory  committee. 
The  foundation  has  arranged  for  expert  testimony  on  behalf  of  patients  who  were 
sued  by  physicians  for  failure  to  pay  claims  which  the   foundation  has  disal- 
lowed.    The  National  Capitol  Medical  Foundation  also  includes  members  of  the 
public  (currently  three  non-physicians)  on  its  board  of  trustees.     The  Utah 
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Professional  Review  Organization  has  a  consumer  advisory  committee  which 
educates  consumers  about  professional  review  activities,  publicizes  the 
program,  encourages  additional   insured  groups  to  use  UPRO  review,  and  con- 
siders such  matters  as  confidentiality  of  information. 


LONGEVITY 


Most  of  the  programs  reviewed  have  been  in  existence  for  several  years,  since 
it  was  anticipated  that  the  potential  for  impact  was  greater  than  in  newer  pro- 
grams.    The  actual  amount  of  operating  experience  is  difficult  to  determine, 
however,  since  the  transition  from  development  to  full  operation  probably  oc- 
curred over  a  period  of  time.     Some  are  still  not  fully  operational. 


Several  programs  have  quite  extensive  experience.     For  example,   the  San  Joaquin 
Foundation  for  Medical  Care  was  created  in  1953  and  has  been  conducting  manual 
review  activities  since  1960.     The  Multonomah  Foundation  for  Medical  Care  was 
created  in  1961;  however,  quality  assurance  activities  did  not  get  under  way 
until  a  reorganization  in  1970.     The  Foundation  for  Health  Care  Evaluation  was 
incorporated  in  1969  and  gained  considerable  experience  in  claims  review  be- 
fore its  conversion  to  a  PSRO.     Most  newer  programs  are  located  within  prepaid 
health  plans.     For  the  most  part,  these  are  oriented  toward  research  and  are 
not  yet  incorporated  into  an  ongoing  review  system. 


In  addition  to  relatively  lengthy  experience,  most  of  these  programs  have  also 
benefited  from  substantial  grant  and  contract  support.     Some  were  initially 
funded  by  the  federal  government  as  Experimental  Medical  Care  Review  Organiza- 
tions (EMCROs),  a  demonstration  program  to  develop  different  methods  for  as- 
sessing the  quality  of  medical  care.   [1]     Others  received  special  grants  or 
state  or  local  support.     It  was  not  possible  to  accumulate  exact  totals  for 
the  amount  of  developmental  support,  but  in  many  cases  it  amounts  to  more  than 
a  million  dollars.     If  the  review  mechanisms  developed  by  these  programs  are 
readily  transferrable ,  newer  programs  may  require  shorter  developmental  phases. 
Nevertheless,  the  sizable  financial   investment  and  operating  experience  of  the 
programs  reviewed  here  stand  in  contrast  with  the  resources  available  to  more 
recently  initiated  PSROs.     Whether  this  leads  to  comparable  differences  in 
effectiveness  is  not  known. 


HOSPITAL  REVIEW  PROGRAMS 

Hospital  review  programs,  both  those  conducted  independently  of  PSROs  and  those 
included  within  a  PSRO,  consist  primarily  of  concurrent  review  and  medical  care 
evaluation  studies.     Retrospective  profile  analysis  of  patterns  of  care  is  a 
third  review  required  by  PSROs,  but  in  most  programs  visited,  this  form  of  re- 
view is  still  being  developed. 


Concurrent  Review 

The  initial  step  in  concurrent  review  is  admission  certification,  usually 
within  24  to  48  hours  of  admission,  which  assures  that  hospital  admission 
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is  medically  necessary.     Medical  necessity  is  determined  by  referring  to 
criteria  developed   for  particular  problems,  diagnoses,  or  procedures  or 
to  more  general  criteria  which  specify  the  types  of  services  which  should 
be  provided  in  a  hosptital.     PSRO  guidelines  were  developed  with  the  ex- 
pectation that  with  more  experience  and  analysis,   it  will  be  possible  to 
identify  specific  patients,  conditions,  or  physicians  with  either  increased 
or  decreased  likelihood  of  inappropriate  admissions.     The  latter  might  be 
automatically  certified,  while  the   former  may  require  re-admission  certi- 
fication.    Most  programs  visited  had  not  reached  this  point,  however. 

When  an  admission  is  certified,  a  length  of  stay  is  assigned  based  on  the 
patient's  condition  and  the  date  when  hospitalization  should  no  longer  be 
required  or  when  review  of  the  necessity  for  continued  stay  would  be  ap- 
propriate.    Reimbursement  is  approved  for  the  initial  stay.     The  assignment 
is  usually  based  on  either  the  50th  percentile  of  lengths  of  stay  for  similar 
patients  (which  may  be  obtained  from  regional  data  provided  by  discharge  ab- 
stract services)  or  on  more  general  level  of  care  criteria. 

Continued  stay  review  is  the  final  step  in  concurrent  review.     At  the  expira- 
tion of  the  initially  assigned  length  of  stay,  records  are  reviewed  to  determine 
whether  continued  stay  is  necessary  and,  sometimes,   to  monitor  selected  aspects 
of  the  quality  of  care.     Justification  for  an  extended  stay  may  be  determined  by 
referring  to  any  of  three  types  of  criteria:     indications  for  discharge,  speci- 
fied services  which  require  hospital  care,  or  the  provision  or  planned  provi- 
sion of  critical  diagnostic  or  therapeutic  services  consonant  with  a  particular 
diagnosis  or  condition.     If  continued  stay  is  certified,  another  length  of  stay 
is  assigned  and  the  process  is  repeated.     The  information  obtained  during  the 
review  process  is  used  to  assist  in  discharge  planning. 

Typically,  both  admission  certification  and  continued  stay  review  are  performed 
by  a  review  coordinator,  usually  a  nurse,  who  has  authority  for  approval  only. 
When  questions  arise,  the  case  is  referred  to  a  physician  advisor  who  has  au- 
thority to  make  adverse  decisions.     In  the  case  of  PSRO  review,  either  the  at- 
tending physician  or  patient  may  appeal  a  decision  to  the  state  level  and,  ul- 
timately, to  the  national  level. 

Although  the  general  pattern  is  described  above,  variations  are  apparent  in  the 
timing,  depth,  and  frequency  of  review.     The  Medical  Care  Foundation  of 
Sacramento  conducts  pre-admission  certification  of  elective  hospital  admis- 
sions, and  in  other  programs  that  option  is  available.     Mt .   Sinai  Hospital 
in  New  York  City  requires  pre-admission  screening  of  elective  admissions 
using  explicit  criteria  to  ensure  that  hospitalization  is  necessary,  that  as 
many  procedures  as  possible  are  performed  outside  the  hospital,  and  that  an 
appropriate  treatment  plan  has  been  developed.     The  pre-admission  form  includes 
socio-demographic   information  which  is  forwarded  to  the  social  services  depart- 
ment,  so  that  discharge  planning  can  begin  prior  to  admission  when  appropriate. 
The  Utah  Professional  Review  Organization  does  not  conduct  pre-admission  certifi- 
cation and  is  not  convinced  of  its  value.     However,  the  state  Medicaid  agency 
requires  prior  approval  for  certain  restorative  or  elective  procedures,  which 
may  reduce  the  volume  of  questionable  admissions  for  UPRO  review.     The  Colorado 
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Foundation  for  Medical  Care  included  pre-admission  certification  for  elective 
admissions  during  a  developmental  phase  of  its  hospital  review  program,  but 
it  is  no  longer  required,  although  two  or  three  hospitals  continue  to  use  it. 
Pre-admission  certification  was  discontinued  because  of  the  reported  inability 
of  most  hospital  organizational  structures  to  accommodate  the  prior  approval 
process  and  requisite  paperwork. 

The  comprehensiveness  of  admission  certification  varies.  Bethesda-Lutheran 
Hospital  uses  an  adaptation  of  the  Commission  on  Professional  and  Hospital 
Activities'  Concurrent  Review  Screening  procedure,  which  assigns  admissions 
to  the  following  categories:     diagnosis  alone  justifies  admission,  and  certi- 
fication is  automatic;  diagnosis  plus  a  scheduled  surgery  or  hospital  proce- 
dure justifies  admission  and,  after  confirming  the  scheduling  of  the  surgery 
or  procedure,  certification  is  automatic;  or,  diagnosis  for  which  symptoms  and 
problems  are  compared  to  predetermined  criteria  reflecting  the  level  of  care 
required.     This  procedure  has  also  been  adopted  by  the  Foundation  for  Health 
Care  Evaluation. 

In  other  programs,  all  admissions  are  reviewed,  but  with  varying  degrees  of 
stringency.     At  the  time  of  the  Colorado  site  visit,  for  example,  there  were 
no  explicit  criteria  for  judging  the  necessity  of  admissions.     The  nurse  coordi- 
nator relied  on  her  judgment,  and  questionable  cases  were  referred  to  the  phy- 
sician reviewer.     Greater  emphasis  was  then  placed  on  reviewing  the  medical 
record  to  ascertain  whether  the  admission  diagnosis  was  validated.     In  May  of 
1976,  however,  explicit  screening  guidelines  were  implemented  in  all  major 
Colorado  acute  care  facilities. 

There  are  similar  variations  in  continued  stay  review.     The  Utah  review  program 
is  one  of  the  more  intensive  and  concentrates  on  facilitating  the  provision  of 
services  throughout  the  entire  hospital  stay,  rather  than  intermittently  assess- 
ing the  need  for  admission  and  continued  stay.     Sets  of  explicit  guidelines  have 
been  developed  for  quality  care,  admission  indications,  and  length  of  stay;  how- 
ever, they  are  not  routinely  applied  to  individual  cases,  but  are  more  likely  to 
be  used  for  profiles  or  quality  assessment  studies.     For  routine  review,  nurse 
coordinators  usually  rely  on  personal  judgment,  supplemented  by  guidelines  for 
level  of  care  and  indications  for  denial  of  admission. 

The  review  program  at  Mt .   Sinai  Hospital   includes  admission  certification  and 
continued  stay  review,  based  on  the  50th  percentile  of  length  of  stay  data  from 
the  New  York  State  Hospital  Utilization  Review  (NYSHUR)  program.     The  utiliza- 
tion review  coordinators  are  expected  to  alert  the  medical  staff  when  inappro- 
priate quality  is  suspected  and  also  are  responsible  for  coordinating  their 
activities  with  other  hospital  departments.     For  example,  they  meet  bi-weekly 
with  the  social  services  department  and  weekly  with  the  home  care  staff,  ad- 
missions office,  and  medical  care  evaluation  committee.     Their  approach  to 
utilization  review  is  interdisciplinary  and  appears  to  be   firmly  established 
throughout  the  hospital. 

At  Overlook  Hospital  there  is  no  admission  certification  or  assignment  of  length 
of  stay.     Instead,   two  utilization  review  coordinators  review  the  care  provided 
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to  all  medical  and  surgical  patients,  as  well  as  those  in  the  80-bed  extended 
care  unit,  once  per  week,  primarily  to  determine  whether  the  level  of  care  is 
appropriate.  For  about  75  percent  of  the  patients,  the  coordinator's  assess- 
ment is  based  on  a  review  of  a  Kardex  file  maintained  in  the  nursing  station, 
which  lists  diagnosis  and  physician's  orders.  For  the  rest  of  the  patients 
a  more  detailed  review  of  the  medical  record  might  be  made. 

In  no  program  visited  was  there  extensive  routine  use  of  explicit  review 
criteria,  with  the  possible  exception  of  the  recent  changes  in  Colorado. 

The  extent  to  which  review  coordinators  and  physician  advisors  are  trained 
and  supervised  also  varies.     Within  the   foundations  and  PSROs ,  most  nurse 
coordinators  are  hired,  trained,  and  paid  by  the  foundation  or  PSRO.  Ex- 
ceptions may  be  made  for  small  outlying  hospitals  where  the  patient  load  may 
not  justify  a  full-time  coordinator.     In  these  hospitals  the  foundation  may 
contract  for  a  percentage  of  the  time  of  a  regular  hospital  nurse. 

As  an  example,   the  Colorado  Foundation  provides  a  three-week  training  program 
for  nurse  coordinators,  which  includes  both  conceptual  and  applied  content. 
A  foundation  supervisor  oversees  the  work  of  the  hospital  coordinators  and, 
in  the  case  of  rural  hospitals,  makes  a  bi-monthly  visit  and  simultaneously 
replicates  the  work  of  the  hospital's  nurse  coordinator.     Foundation  officials 
are  concerned  that  after  a  period  of  time,  coordinators  may  identify  more  with 
the  hospital  to  which  they  are  assigned  than  with  the  foundation.     They  are 
considering  rotating  coordinators  among  hospitals,  but  there  is  currently 
no  consistent  rotation  policy.  The  Sacramento  Foundation  has  already  adopted 
a  policy  of  rotating  nurse  coordinators  every  eight  to  12  months.     As  hospi- 
tals receive  delegated  status  under  PSROs,  nurse  coordinators  most  likely 
will  become  hospital  employees,  which  may  increase  mixed  loyalties. 

Arrangements  for  physician  advisors  are  even  more  varied.     At  the  Sacramento 
Foundation,  physician  advisors  are  appointed  by  the  foundation's  board  of 
directors  and  paid  by  the   foundation.     A  review  manual  or  set  of  guidelines 
is  provided,  although  it  is  not  always  used   in  the  review  process.     Some  PSROs 
which  do  not  officially  delegate  responsibility  for  review  to  hospitals  still 
permit  the  hospital  to  select  and  reimburse  the  physician  advisors.     The  PSRO 
usually  retains  formal  authority  to  appoint  the  advisor,  but  the  hospital's 
nomination  is  seldom  overridden.     Nevertheless,  the  physician  advisors  are 
specifically  identified  and  officially  involved  in  many  functions,  usually 
including  membership  on  advisory  committees,  where  they  exchange  experiences 
with  colleagues  from  other  hospitals  within  the  PSRO  area.     If  the  hospital  has 
delegated  status,  the  physician  advisor  would  be  more  closely  identified  with 
the  hospital.     The  San  Joaquin  PSRO  combines  standardized  policies   for  the 
nurse  coordinator  with  more  flexible  procedures   for  the  physician  advisor. 
Nine  of  the  13  hospitals  have  delegated  responsibility  for  appointing  phy- 
sician advisors  and  conducting  MCEs .     Those  hospitals  have  lists  of  physicians 
on  whom  the  nurse  coordinator  may  call  when  questions  arise.     In  non-delegated 
hospitals  the  nurse  coordinator  can  call  any  of  about   150  physicians,  depending 
on  specialty.     (The  San  Joaquin  PSRO  has  about  340  physician  members.)  Phy- 
sicians receive  a  one-page  summary  of  review  procedures  when  they  are  asked  to 
volunteer  for  review  functions;  volunteers  may  receive  additional  assistance 
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from  the  nurse  coordinator.     In  responding  to  questions  about  appropriateness 
of  admission  or  continued  stay,   the  physician  advisor  apparently  relies  pri- 
marily on  personal  professional  judgment  except  for  a  few  procedures,   such  as 
ileal  by-pass,   for  which  explicit  criteria  have  been  developed. 


Medical  Care  Evaluation  Studies  (MCEs) 

Under  PSRO,  medical  care  evaluation  studies  are  a  "form  of  health  care  review 
in  which  an  in-depth  assessment  is  made  of  the  quality  and  administration  of 
health  care  services"  to  assure  that  services  are  "appropriate  to  the  patient's 
needs  and  are  of  appropriate  quality,   and... health  care  organization  and  admin- 
istration support  the  timely  provision  of  quality  care."   [2]     MCEs  may  be  per- 
formed within  a  single  hospital,  within  a  department  of  a  hospital,   or  within 
a  group  of  hospitals.     Results  of  MCEs  are  intended  to  lead  to  improvement, 
either  through  administrative  changes  or  continuing  medical  education.  This 
detailed,   frequently  retrospective  assessment  of  the  care  given  to  groups  of 
patients  stands  in  contrast  to  concurrent  review  of  individual  patients. 

The  potential  subject  matter  of  an  MCE  is  broad  and  could  include  a  "review 
of  the  patient  care  process,  patient  outcomes,   the  use  of  a  given  procedure, 
or  the  operating  characteristics  of  an  institution."   [3]     An  MCE  should  not 
be  equated  with  clinical  research,  however,  and  is  confined  to  diagnoses  and 
therapies  for  which  there  are  well  established  criteria  for  appropriate  prac- 
tice.    An  example  of  an  MCE  objective  would  be  to  assure  that  outcomes  of 
patients  with  pneumococcal  pneumonia  are  acceptable. 

PSRO  requirements  for  MCEs  are  intentionally  general  so  that  traditional  medi- 
cal audit  activities  can  be  included,   as  well  as  programs  such  as  the  JCAH's 
Performance  Evaluation  Procedure  for  Auditing  and  Improving  Patient  Care  (PEP) 
and  the  California  Medical  Association's  Patient  Care  Audit  (PCA),  which  are 
usually  more  specific.     For  example,   the  PEP  system  requires  the  specification 
of  criteria  with  100  or  zero  percent  compliance  standards;  the  PCA  format  in- 
cludes a  "threshold  of  action,"  usually  a  specified  percentage  of  aberrant 
cases  which  indicates  that  remedial  action  should  be  initiated.     A  comparison 
of  PSRO,  PEP,   and  PCA  requirements  is  included  in  Appendix  D. 

The  intent  of  the  national  PSRO  guidelines  is  to  link  the  three  review  com- 
ponents into  an  integrated  review  system.     Ideally,   concurrent  review  and  MCEs 
should  reinforce  each  other.     Similarly,  profile  analysis  could  identify  areas 
for  increased  or  decreased  attention  for  concurrent  review  and  MCEs,  and  moni- 
tor their  effectiveness. 

Most  PSRO  administrators  concentrated  their  initial  efforts  on  the  concurrent 
review  process  and  are  just  now  beginning  to  develop  and  implement  policies 
for  MCEs,  whereas  hospital  programs  pre-dating  PSROs  place  more  value  on  MCEs 
than  on  concurrent  review.     At  the  Sacramento  and  San  Joaquin  PSROs,   in  parti- 
cular, hospitals  are  permitted  to  continue  with  whatever  medical  audit  activ- 
ities they  had  traditionally  done,  pending  development  of  the  final  PSRO  re- 
quirements.    Among  the  programs  visited,  delegation  to  hospitals  of  responsi- 
bility for  MCEs  appears  to  be  more  likely  than  for  concurrent  review. 
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The  Multnomah  Foundation  has  already  delegated  MCE  responsibilities  sep- 
erately  from  concurrent  review.     At  the  time  of  the  site  visit  the  Colorado 
Foundation  conducted  a  non-delegated  review  system.     By  the  end  of  September 
1976,  however,  84  of  93  hospitals  have  received  delegated  status  for  MCEs . 
Five  Colorado  hospitals  have  delegated  responsibility  for  concurrent  review. 
In  one  hospital  visited,  the  non-delegated  concurrent  review  activities  and 
resulting  documentation  were  conducted  independently  from  the  hospital's 
utilization  review  committee.     No  information  was  obtained  on  the  medical 
audit  process,  but  it  apparently  was  not  integrated  with  concurrent  review. 
Within  the  Colorado  Foundation  responsibilities  for  concurrent  review,  MCEs, 
and  profile  analysis  are  departmentalized,  which  may  encourage  fragmentation, 
although  more  recent  information  from  the  foundation  indicates  an  effort  to 
integrate  review  components.     Review  programs  are  developed  at  the  central 
office  and  implemented  on  a  regional  basis. 

Some  programs  have  conducted  MCEs,  either  areawide  or  within  individual  hospi- 
tals.    The  Foundation  for  Health  Care  Evaluation  has  conducted  at  least  two 
areawide  audits.     These  were  preliminary  efforts  to  test  methods  for  criteria 
development,  data  gathering,  and  analysis,  and  did  not  include  the  total  pro- 
cess of  education  and  re-assessment.     Areawide  audits  are  planned  by  the 
National  Capital  Medical  Foundation.     More  extensive  experience  has  been  accu- 
mulated in  Utah.     In  all  places  visited  common  problems  were  encountered,  in- 
cluding the  incompatibility  between  JCAH  and  PSRO  requirements;  the  difficulty 
of  selecting  audit  topics  which  result  in  the  identification  of  significant 
problems  so  that  improvement  can  be  made;  the  difficulty  of  developing  cri- 
teria relevant  for  all  patients  but  which  do  not  become  too  general;  and  the 
difficulty  of  achieving  change,  once  deficiencies  have  been  identified.  Some 
innovative  attempts  to  deal  with  these  problems  were  also  explored. 

The  difficulty  of  identifying  "real"  problems  for  medical  audit  was  mentioned 
above.     Common  methods  for  selecting  topics  are  to  concentrate  on  the  ten  most 
frequently  occurring  diagnoses  within  a  hospital  or  to  cover  all  diagnoses 
which  account  for  a  certain  percentage  of  admissions.     These  approaches  auto- 
matically exclude  cases  with  non-specific  diagnoses  and  ignore  certain  proce- 
dures which  may  affect  all  patients,  regardless  of  diagnosis. 

The  Multnomah  Foundation  staff  expressed  concern  with  the  usefulness  of  tradi- 
tional medical  audit.     They  are  beginning  to  monitor  MCEs  and  are  developing 
a  mechanism  to  review  hospital-initiated  MCEs  at  the  protocol  design  and  com- 
pletion stages.     As  a  way  of  identifying  potential  MCE  topics  which  should  lead 
to  improvements  in  patient  care,  the  possibility  of  instituting  a  "problem  pro- 
cessing" system  is  being  explored.     As  planned,  the  system  would  permit  anyone 
connected  with  a  hospital  or  quality  assurance  program  to  note  perceived  defi- 
ciencies in  the  patient  care  process  which  might  be  worthy  of  audit.     A  stan- 
dard form  would  be  distributed  throughout  the  hospital  for  recording  such  in- 
formation, which  would  be  forwarded  to  the  review  coordinator  or  foundation 
staff  for  analysis.     Over  time,  problems  would  be  accumulated.     If  patterns 
emerged,  they  would  then  become  the  subject  of  an  MCE. 

Bethesda-Lutheran  Hospital  staff  has  had  extensive  experience  with  medical 
audit,  beginning  in  1968,  and  has  encountered  similar  problems.  Initially, 
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efforts  were  made  to  develop  criteria  for  specific  diagnoses  and  to  conduct 
audits  for  all  diagnoses  which  accounted  for  two  percent  or  more  of  total  hos- 
pital admissions.     About  20  months  were  required  to  complete  the  total  review 
cycle;  nevertheless,  many  patients  were  never  reviewed  because  they  were  not 
included  among  the  diagnoses  which  constituted  two  percent  or  more  of  the  to- 
tal admissions.     To  overcome  these  problems,  the  patient  care  monitor  (PCM) 
was  developed.     The  PCM  is  a  data  retrieval  mechanism  which  routinely  gathers 
basic  information  on  all  patients,  including  history  and  physical,  lab  tests 
and  procedures,   transfusion  practices,  and  complications.     The  data  are  ana- 
lyzed quarterly  within  and  across  diagnoses.     Identified  problems  are  assigned 
on  a  priority  by  the  quality  assurance  staff  and  may  become  the  topic  of  an 
MCE.     Use  of  the  PCM  has  greatly  facilitated  the   identification  of  signifi- 
cant problems   for  audit. 

Bethesda-Lutheran 1 s  PCM  is  not  unlike  the  PSRO  requirements  for  profile  ana- 
lysis, except  that  the  PCM  appears  to  include  more  information.     The  experience 
at  Bethesda-Lutheran  suggests  that  as  profile  analyses  are  more  routinely  per- 
formed, the  difficulties  in  selecting  MCE  topics  may  be  partially  alleviated. 

The  Comprehensive  Quality  Assurance  System  (CQAS)  of  Kaiser  Permanente  in 
Northern  California,  Denver,  and  Cleveland  constitutes  a  pragmatic  approach 
to  medical  care  evaluation  that  is  not  grounded  in  a  systematic  coverage  of 
all  diagnoses  or  any  kind  of  baseline  information.   [4]     Instead,  CQAS  concen- 
trates on  specific  problems  found  to  exist — either  by  an  informal  review  of  a 
small  number  of  medical  records  or  professional  opinion.     The  CQAS  philosophy 
suggests  that  any  number  of  probable  patient  care  problems  can  be  readily  iden- 
tified and  that  it  is  a  better  use  of  resources  to  concentrate  on  them,  rather 
than  to  waste  time  attempting  to  analyze  the  total  spectrum  of  care.     CQAS  is 
described  in  more  detail  below  as  an  ambulatory  care  review  system.     For  pur- 
poses of  this  section,  however,  it  may  be  regarded  as  an  alternative  approach 
to  medical  care  evaluation. 

Another  recurring  problem  is  the  amount  of  time  required  to  generate  criteria 
for  MCEs  and  the  difficulty  of  specifying  criteria  which  are  appropriate  for 
the  wide  range  of  variations  which  may  occur  within  a  single  diagnostic  or 
problem  category.     An  approach  called  "criteria  mapping,"  or  MAPS,  has  been 
developed,  which  incorporates  branching  techniques  and  attempts  to  replicate 
the  clinical  decision-making  process  in  a  manner  which  would  permit  assessing 
the  quality  or  appropriateness  of  care  provided  to  an  individual  patient  at  a 
particular  point  in  time.   [5]     It  builds  on  earlier  work  using  protocols  or 
algorithms  to  train  physician  extenders.  [6] 

In  using  the  MAPS  approach,  criteria  are  developed  by  physicians  to  reflect 
medical  logic  and  cover  the  entire  spectrum  of  care,  ranging  from  an  initial 
ambulatory  contact   to  potential  hospitalization.     Criteria  are  grouped  accord- 
ing to  patient  care  objectives,  and  the  results  or  findings  at  each  step  lead 
to  subsequent  decisions  or  actions.     The  criteria  are  put   into  a  map  or  branch- 
ing format  and  applied  by  non-physicians  in  conducting  retrospective  audits  of 
the  medical  record.     Only  relevant  criteria  are  applied  to  any  individual  case. 
The  data  are  computerized,  and  findings  permit  the  determination  of  individual 
case  scores  based  on  compliance  with  applicable  criteria  and  the  degree  of 
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compliance  of  all  applicable  cases  with  specific  criteria.     Existing  maps  have 
been  tested  for  reliability  and  transferability.     Training  manuals  and  ab- 
stracting guidelines  are  available.     The   investigators  believe  that  their 
technique  is  probably  most  appropriate  for  complex  decision-making  problems 
or  the  treatment  of  very  complicated  diagnoses.     For  simpler  conditions,  where 
patient  characteristics  have  little   influence  on  treatment,  mapping  may  not 
be  worthwhile. 

The  other  major  problem  identified  with  MCEs   is  the  difficulty  of  altering 
provider  behavior   to  correct   identified  deficiencies.     The  Utah  Professional 
Review  Organization  has  had  extensive  experience  with  traditional  medical 
audit,  has  summarized  the  deficiencies,  and  is  testing  new  approaches.  Mini- 
mal behavior  change  resulted   from  about   20  traditional  medical  audits  con- 
ducted during  an  18-month  period,  beginning  in  December  of  1972.     The  extensive 
process-oriented  data  gathered  could  be  categorized  as  follows:     criteria  with 
which  there  was  high  compliance,  criteria  which  the  physicians  believed  were 
of  questionable  validity,  and  the  very  few  criteria  which  actually  influence 
outcome.     The  last  category  was  further  complicated  by  the  difficulty  of 
attempting  to  change  behavior  retrospectively.     More  specifically,  by  the 
time  that  the  MCE  data  were  gathered,  analyzed,  and  incorporated  into  an 
educational  activity,  the  patient  had  long  since  been  discharged  from  the 
hospital — usually  with  a  positive  outcome — and  the  need  for  changed  behavior 
was  not  immediately  apparent. 

To  overcome  these  deficiencies  in  traditional  audit,  UPRO  is  experimenting 
with  new  approaches.     One  emphasis  is   to  reduce  the  amount  of  process  data 
gathered  and  concentrate  on  a  few  procedures  which  obviously  influence  out- 
come and  would  be  harder  to  ignore,   such  as  treatment  of  patients  with  cata- 
racts, the  extent  of  fetal  monitoring  in  induced  obstetrical  deliveries,  use 
of  whole  blood   instead  of  packed  cells,  or  failure   to  work-up  patients  with 
anemia.     Special  studies  are  being  conducted  to  assess  the  achievement  of  a 
few,  critical  "management  objectives,"  which  are  statements  of  the  objectives 
of  the  treatment  process  for  a  particular  diagnosis  or  problem  by  the  time 
of  discharge.     Objectives  have  been  written  for  14  diagnoses.     As  an  example, 
management  objectives  for  a  patient  with  appendicitis  are:     ambulatory,  afe- 
brile (temperature  below  99  degrees),   and  able   to  maintain  nutrition  orally. 
Finally,  UPRO  staff  hopes  to  conduct  some  concurrent  MCEs  which  would  permit 
intervention  during  treatment  if  necessary  to  improve  care. 

LONG-TERM  CARE  REVIEW 

Of  the  programs  visited,  only  the  Multnomah  and  Sacramento  foundations  and 
the  San  Joaquin  PSRO  review  the  quality  of  long-term  care.     The  emphasis  in 
these  programs  is  similar  to  the  review  of  acute  care;  sometimes  the  same 
staff  review  both.     A  study  of  long-term  care   in  Colorado,  that  also  attempted 
to  develop  assessment  instruments,  has  been  conducted  under  the  sponsorship 
of  the  Colorado  Foundation.     Although  the  foundation  was  not  reviewing  long- 
term  care  at  the  time  of  the  site  visit,   it  will  begin  a  demonstration  pro- 
ject  in  October  1976.     The  project   includes  the  development  of  improved 
documentation  for  patient  transfers  among  facilities,  an  exploration  of  the 
need   for  alternative  lower  levels  of  care  providing  supportive  social  services, 
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and  the  testing  of  criteria  based  on  functional  level  and  problems  rather 
than  on  diagnoses  alone. 

The  limited  activities  elsewhere  may  partially  stem  from  the   fact  that  PSRO 
draft  guidelines  for  long-term  care  review  have  only  recently  been  discussed 
by  the  National  Professional  Standards  Review  Council  and  not  yet  issued  to 
local  PSROs.     PSRO  review  for  both  acute  and  long-term  care  emphasizes  the 
medical  necessity  and  appropriateness  of  admission  and  continued  stay  and 
requires  medical  care  evaluation  studies.     According  to  the  draft  long-term 
care  guidelines,  however,  there  also  are  important  differences. 

Pre-admission  certification  will  be  required  for  long-term  care,  either  by 
the  hospital  review  staff  (when  a  patient  is  discharged  from  a  hospital  to 
a  long-term  care  setting)  or  by  independent  agencies  such  as  geriatric  assess- 
ment centers  or  community  mental  health  centers  (if  the  patient  does  not  enter 
from  an  acute  care  hospital).     In  addition  to  the  usual  elements  of  concurrent 
review,  the  long-term  care  requirements  include  a  review  of  the  health  services 
provided  to  each  patient  and  the  extent  to  which  they  meet  the  patient's  needs 
or  are  "provided  in  a  manner  consistent  with  local  standards  of  care."  [7] 
Concurrent  quality  assessment  will  be  performed  on  site,  based  primarily  on  the 
medical  record.     At  least  one  yearly  bedside  review  of  the  patient  is  also  re- 
quired.    Delegation  of  long-term  care  review  responsibility  is  expected  to  be 
less  frequent  than  in  acute  settings.     A  multidisciplinary  group  of  providers 
must  be  involved  in  all  aspects  of  long-term  care  review. 

Because  most  long-term  care  review  activities  were  still  being  developed,  they 
were  not  examined  in  detail  during  the  site  visits.  They  are  the  subject  of  a 
separate  paper,  however,  which  is  summarized  in  Chapter  5. 


AMBULATORY  CARE  REVIEW 

Ambulatory  care  review  programs  can  be  categorized   into  either  ongoing  claims 
review  systems  or  different  approaches  which  generally  evolved,  or  are  evolv- 
ing from  research  activities  or  special  projects.     Both  categories  are  discussed 
below  and  additional  issues  relating  to  quality  assurance  for  ambulatory  care 
are  summarized   in  Chapter  5. 


Claims  Review  Programs 

Ambulatory  quality  assurance  programs  based  on  claims  review  are  conducted 
for  or  in  conjunction  with  third  party  carriers.     Occasionally,  the  review 
organization  is  the  insurance  carrier.     After  a  claim  has  been  submitted  for 
payment,  review  may  include  such  activities  as  ascertaining  that  the  patient 
was  eligible  for  coverage,  that  the  services  provided  were  among  those  in- 
cluded in  the  insurance  benefit  package,  that  any  deductibles  or  coinsurance 
requirements  were  met,  that  the  claimed  fee  on  the  relative  value  scale  was 
commensurate  with  services  rendered,  and  that  the  services  were  not  clearly 
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inappropriate  for  the  diagnosis  or  condition.     The  quality  assurance  programs 
visited  vary  in  the  extent  to  which  they  conduct  these   functions,  although  only 
the  latter  function  is  directly  related  to  the  quality  of  care. 

For  example,  the  San  Joaquin  and  Sacramento  foundations  perform  all  above  func- 
tions for  all  services  provided  to  beneficiaries  of  certain  insurance  carriers. 
The  Colorado  system  for  Medicaid  claims  (discontinued  in  October  1975)  consisted 
of  a  manual  review  of  only  claims  which  had  been  rejected  from  a  computerized 
screen  operated  by  the  fiscal  intermediary  and  based  on  criteria  developed  by 
the  foundation.     The  manual  review  included  an  assessment  by  non-physicians  of 
the  content  of  the  visit  to  determine  the  appropriateness  of  the  relative  value 
scale  coding  and  a  comparison  of  content  and  frequency  of  visits  with  previously 
defined  criteria  to  evaluate  the  appropriateness  of  care.     Questionable  cases 
were  referred  to  physician  reviewers  for  analysis.     Until  recently,  the  Utah 
Physicians  Ambulatory  Care  Evaluation  (PACE)  program  was  not  involved  in  the 
payment  process  and  concentrated  its  review  of  Medicaid  claims  exclusively  on 
questions  of  medical  appropriateness. 

Most  claims  review  programs  concentrate  on  over-utilization  (excessive  number 
of  visits  for  a  single  diagnosis;  prescribed  drugs  or  injections  not  indicated 
by  diagnosis;  or  multiple  visits  to  multiple  providers).  However,  the  profile 
analysis  components  of  the  PACE  and  San  Joaquin  Patterns  of  Treatment  programs 
also  permit  the  detection  of  under-utilization  by  considering  delays  in  estab- 
lishing a  diagnosis,  or  the  absence  of  diagnostic  and  therapeutic  procedures 
indicated  by  the  reported  diagnosis. 

The  extent  to  which  review  by  explicit  criteria  is  computerized  also  varies. 
The  Colorado  review  for  commercial  carriers,  as  well  as  the  Sacramento  and 
current  San  Joaquin  programs,  are  manual  operations.     As  noted  above,  the 
discontinued  Colorado  Medicaid  review  program  was  computerized  at  the  fiscal 
intermediary  level,  although  the  foundation's  subsequent  involvement  was  manual. 
The  New  Mexico  Medicaid  review  system  is  one  of  the  more  comprehensive  computer 
reviews  and  includes  hospital  and  nursing  home  stays,  surgical  services,  office 
visits,  emergency  room  visits,  outpatient  visits,  laboratory  tests,  injections, 
X-rays,  prescriptions,  and  "other."     It  was  not  visited  during  this  study,  but 
has  been  extensively  described  and  evaluated  elsewhere.  [8] 

The  Patterns  of  Treatment  (formerly  Model  Treatment)  program,  developed  by  the 
San  Joaquin  Foundation  for  Medical  Care,   is  a  computerized  review.     It  was  used 
to  review  ambulatory  care  received  by  47,000  Medi-Cal  recipients.     Although  it 
is  no  longer  used  in  California,   it  is  being  tested  in  two  other  sites.  Re- 
view criteria  were  developed  by  the  data  committee  of  the  United  Foundations 
for  Medical  Care  in  consultation  with  several  speciality  colleges. 

Claims  subjected  to  the  Patterns  of  Treatment  program  are   initially  processed 
through  an  administrative  review,  which  checks   for  patient  eligibility  and  such 
errors  as  incorrect  procedure  or  provider  number.     Claims  which  pass  this  re- 
view are  subjected  to  medical  review  against  the  patient's  claims  history  to 
check  on  duplications  in  drugs  or  other  services.     Claims  are  then  submitted 
to  the  "broad  screen"  which  eliminates  from  further  consideration  and  pays 
claims  for  initial  visits  involving  minimal  sums  of  money,  or  remands  for 
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manual  review  those  claims  related  to  rare  disorders  for  which  criteria  have 
not  been  developed.     Claims  which  do  not  pass  the  broad  screen  enter  the  Pat- 
terns of  Treatment  program,  which  specifies  recommended  amounts  and  types  of 
procedures  and  prescriptions  for  56  diagnostic  groupings.     Claims  which  pass 
this  portion  of  review  are  paid.     Those  which  do  not  are  reviewed  by  a  physi- 
cian, along  with  a  patient  profile  and  any  other  information  requested  from 
the  attending  physician. 

The  Utah  Physician  Ambulatory  Care  Evaluation  (PACE)  program  was  initiated  to 
test  further  the  extent   to  which  claims  information  can  be  used  for  quality 
assessment,  provided  there  is  a  high  volume  of  claims  and  the  system  can  gen- 
erate both  physician  and  patient  profiles.     Originally,  PACE  had  the  capacity 
to  provide  additional  information  on  an  on-line  basis,  using  a  cathode  ray 
terminal,  but  this  feature  is  no  longer  operational. 

PACE  has  been  linked  to  the  Utah  Medicaid  Management  Information  System  (MMIS) 
The  state  Medicaid  office  provides  UPRO  with  a  computer  tape  containing  claims 
information  from  physicians,  other  health  professionals  whose  services  are  cov 
ered  by  Medicaid,  and  pharmacies.     The  tape  is  processed  through  the  PACE 
screen;  exceptions  are  remanded  for  physician  review.     Screening  guidelines 
were  developed  by  specialty  panels  and  consist  of  combinations  of  diagnoses, 
therapies,  or  investigations  which  are  either  critical  to,  or  inconsistent 
with,   ideal  care.     Criteria  have  also  been  developed  to  link  events  over  a 
specified  time  period.     In  addition  to  reviewing  exceptions,  reviewers  period- 
ically examine  profiles.     Ready  access  to  this  additional  information  has  been 
credited  with  stimulating  the  interest  of  reviewing  physicians  in  the  review 
process.     It  has  also  led  to  the  generation  of  additional  review  criteria,  as 
questions  for  a  particular  case  are  found  to  have  more  general  applicability. 


Other  Ambulatory  Review  Programs 

Within  the  ambulatory  review  programs  wh 
tion,  some  are  not  fully  developed  and, 
briefly.     Others  are  operational  and  may 
review. 


ich  are  not  based  on  claims  informa- 
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be  regarded  as  alternatives  to  claims 


The  Southern  California  Region  of  the  Kaiser-Permanente  Medical  Care  Program 
is  conducting  a  research  project  to  develop  and  implement  a  quality  assurance 
program  which  can  compare  the  Kaiser  medical  centers  and  also  provide  overall 
estimates  of  quality.     Measures  of  input,  process,  outcome,  and  access  to  care 
are  being  developed  for  six  diagnoses  or  conditions  in  both  ambulatory  and  in- 
patient treatment   settings.     The  primary  focus  of  the  study  is  on  health  out- 
comes.    An  attempt  will  be  made  to  isolate  correlates  of  outcomes  to  determine 
whether  short-term  proxy  outcome  measures  can  be  developed.     Data  sources  vary 
depending  on  the  diagnosis,  and  include  both  the  medical  record  and  physical 
examination.     A  questionnaire  has  been  used  to  elicit  patient  characteristics, 
perceptions  of  outcome,  and  satisfaction.     The  study  design  includes  a  calcula 
tion  of  cost  and  time  requirements  for  routine  use  of  these  techniques.  Find- 
ings are  not  yet  available. 
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The  Multnomah  Foundation  is  developing  the  Multi-Use  Medical  Care  Data  System 
(MUMCDS)  which  should  make  available  to  physicians  a  computer  record  of  each 
patient's  personal  characteristics,  insurance  coverage,  and  past  medical  his- 
tory, as  well  as  information  about  each  encounter  in  a  format  that  would  re- 
place much  of  the  conventional  medical  record.  MUMCDS  will  include  criteria 
to  assess  the  process  of  care;  assessments  will  be  provided  to  the  physician 
in  an  educational  context   to  improve  the  quality  of  care. 

An  informed  consent   form  has  been  developed  on  which  a  patient  may  give  writ- 
ten permission  to   include   information  from  his  medical  record   in  MUMCDS.  The 
form  stresses  benefits  to  be  derived   from  MUMCDS  by  simplifying  record  keep- 
ing,  avoiding  duplication,   and  making  more  physician  time  available   for  patient 
care . 

Data  sources  include  a  patient  data  form,  completed  once  unless  there  are 
changes,  and  a  patient  encounter  form  which  covers  all  aspects  of  the  visit 
including  presenting  complaints,   findings,  conditions  treated,  selected  treat- 
ment  procedures,  drugs  prescribed,  diagnostic  procedures  ordered,  counseling, 
follow-up  plan,  and  length  and  place  of  visit. 

A  long  developmental  period  is  expected.     Nevertheless,  MUMCDS  is  intended 
eventually  to  be  used   in  many  different  practice  settings,   including  small 
f ee-f or-service  offices.     With  that  exception  it  resembles  the  Health  Infor- 
mation System  of  the  Indian  Health  Service  in  Tucson,  which  assesses  the  qual- 
ity of  ambulatory  care  provided   to  the  Papago  Indian  tribe. 

The  Health  Information  System  (HIS)  retrieves  information  on  all  ambulatory 
care  provided   to  tribal  members,  regardless  of  site,   including  care  rendered 
in  schools  and  homes  by  public  health  nurses  and  tribal  health  workers,  as 
well  as  care  provided  in  the   four  clinics  and  50-bed  hospital.     Remote  com- 
puter terminals  are  available  for  entering  and  retrieving  information. 

The  executive  health  staff  of  the  tribal  council   is  officially  responsible 
for  selecting  problems  for  assessment.     The  decision  is  based  on  prevalence, 
potential  severity,   and  the  existence  of  effective  treatment.     Two  "maps" 
are  constructed   for  each  problem.     Staging  maps  break  each  health  problem 
into  stages  of  increasing  severity  to  define  areas   for  intervention  and  anti- 
cipate outcome.     Process  maps  define  the  general  steps   in  the  clinical  manage- 
ment of  the  problem  and  usually  include  screening,  diagnostic  work-up,  pre- 
ventive and  remedial  therapy,  and  follow-up.     Finally,  criteria,  standards, 
and   indicators  are  developed  to  be  used  in  the  automated  quality  assessment. 

The  assessment   is  based  on  information  from  encounter  forms  and  lab  slips  and 
consists  of  the   following  steps:     location  within  the  data  base  of  patients  for 
a  particular  audit;  categorization  of  patients  by  priority  status   (a  function 
of  risk  and  prognosis)  based  on  their  histories  and  the  staging  maps;  identi- 
fication of  encounters   for  the  priority  patients  (for  example,   if  the  audit 
concentrated  on  screening,   follow-up  visits  would  be  ignored);  identification 
and  separate  storage  of  audit  information;  and  application  of  the  indicators 
to  the  stored   information  to  determine  effectiveness  of  care. 
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The  system  produces  three  reports:     population-based  reports   to  be  used  by 
system  managers;  provider-based  reports  for  use  by  health  workers   in  self- 
assessment;  and  provider-based  reports  to  be  used  by  supervisors.     It  should 
be  noted  that  this  is  a  closed  health  care  system,  where  providers  have  rec- 
ognized responsibility  for  the  care  of  a  well-defined  and  relatively  stable 
population . 

Another  population-based  review  program  is  the  Comprehensive  Quality  Assurance 
System  (CQAS)  developed  by  the  Kaiser-Permanente  Medical  Care  Program  of 
Northern  California.   [9]     CQAS  is  designed  to   identify  and  examine  instances 
of  less  than  optimal  care  to  bring  about  an  improvement   in  quality.     There  is 
no  concern  with  documenting  overall  levels  of  care  and,   for  that  reason, 
systematic  sampling  techniques  are  not  used.     Instead,   the  selection  of  topics 
and  records  is  deliberate  and  designed  to  yield  the  greatest  possible  number 
of  instances  of  poor  performance. 

In  the  CQAS  system,  records  are  selected  for  review  by  a  process  called  micro- 
sampling,  in  which  small  numbers  of  records  are  analyzed  at   infrequent  inter- 
vals emphasizing  the  times  and  areas  where  performance  is  likely  to  be  poor. 
If  a  CQAS  committee  felt  that  sampling  among  records  or  patients  seen  at  the 
end  of  a  day  would  yield  more  instances  of  inadequate  care,   for  example,  then 
records  would  be  drawn  from  that  group.     Records  are  reviewed  by  a  physician, 
who  identifies  and  records  problems  based  on  personal  judgment.     Records  are 
then  reviewed   independently  by  an  additional  reviewer  and  other  problems  in 
care  are  added.     Unanimous  agreement  is  required  to  assure  that  the  problems 
represent   significant  deficits  in  care.     The  problem  list  then  goes   to  a  commit- 
tee.    Committee  actions  include  standard  setting,  recommendations  for  education, 
imposition  of  sanctions,  changes  in  or  development  of  new  systems,  or  purchasing 
of  equipment.     Standards  are  set  if  the  problem  is  thought  to  occur  often  and 
if  it  could  have  a  serious  impact.     Compliance  with  standards  is  determined 
according  to  the  usual  audit  cycle  of  measurement,  action,  and  re-measurement. 

This  technique  has  been  applied  to  both  hospital  and  ambulatory  care  and  to 
the  full  range  of  providers,  and  appears  to  be  flexible.     It   is  largely  de- 
pendent on  the  medical  record,  and  in  its  early  stages,  on  physician  involve- 
ment.    Improvement  can  be  documented  for  the  specific  types  of  cases  audited, 
if  one  assumes  that  comparable  samples  of  patients  are   included  in  both  mea- 
surement sessions.     However,  overall  performance  cannot  be  assessed. 

The  system's  developers  note  two  problems  in  its  implementation.     Some  CQAS 
committees  have  had  difficulty  establishing  measurable  standards  to  resolve 
identified  problems,  and  technical  assistance  is  sometimes  required.     In  addi- 
tion, the   initial  assumption  that  an  untrained  person  could  retrieve  selected 
medical  records  and  abstract  required   information  has  been  questioned.  CQAS 
officials  now  believe  that   familiarity  with  medical  terminology  and  medical 
audit  are  essential. 

The  EMCRO  project  at  the  Columbia  Maryland  Medical  Plan  has  developed  an  out- 
come oriented  "patient  reported  problem  status  measure"   (PSM),  which  bases  an 
assessment  of  care  on  the  patient's  perception  of  his  health  problems  in  terms 
of  symptoms.    [10]     At  this  point  the  PSM  is  primarily  a  subject  of  research, 
but  it  could  be  used   for  routine  quality  review. 
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In  one  application  of  the  PSM,   patients'  assessments  of  health  problems  were 
obtained  by  questionnaires  completed  one  month  after  treatment   for  either 
sore  throat,   upper  respiratory  infection,  or  urinary  tract  infection.  The 
patient  rated  himself  on  a  scale  from  "none"  to  "extreme"  for  the  following 
items:     frequency  of  symptoms,   intensity  of  symptoms,   activity  limitations, 
and  extent  of  anxiety.     The  patients'  assessments  were  compared  with  outcome 
standards  developed  by  the  medical  staff  to  reflect  the  maximum  symptoma- 
tology expected  for  each  item.     Medical  records  were  reviewed  to  assist  in 
identifying  reasons  for  not  meeting  outcome  standards,  giving  particular 
attention  to  errors  in  the  delivery  of  care  and  patient  characteristics  or 
illnesses.     The  record  review  included  all  patients  with  sub-standard  out- 
comes and  a  30  percent  sample  of  patients  with  acceptable  outcomes,  and 
employed  explicit  process  criteria,  as  well  as  implicit  judgments  of  the  total 
process  of  care.     After  completing  the  process  assessment,  the  reviewer  learned 
the  patient-reported  outcome  and  noted  any  further  actions  which,   if  taken, 
would  have  improved  the  quality  of  care. 

The  detected  deficiencies  in  care  for  the  two  groups  of  patients  were  sig- 
nificantly different.     Of  those  in  the  "acceptable"  group,  56  percent  had  a 
possible  error  in  diagnosis  and  three  percent  had  a  definite  error  in  diag- 
nosis;  the  comparable  numbers  for  the  "sub-standard"  group  were  29  percent 
and  57  percent,  respectively.     Specific  actions  which  could  have  improved 
the  quality  of  care  were  identified  for  22  percent  of  the  total  sample  of 
patients;  of  those,   71  percent  had  sub-standard  outcomes.     Actions  to  im- 
prove quality  were  suggested  for  30  percent  of  the  cases  defined  as  sub- 
standard by  the  process  review  and  for  95  percent  of  the  cases  defined  as 
sub-standard  by  outcome  assessment.     At  least  in  this  application,  then,  an 
outcome  approach  to  assessing  the  quality  of  ambulatory  care  was  more  effi- 
cient than  a  process  review.     Furthermore,   involvement  of  patients  in  assessing 
quality  contributes  information  which  is  not  available  from  the  medical  record 
(especially  if  poor  outcome  stems  from  the  absence  of  a  follow-up  visit)  and 
lessens  demand  on  physician  time,   since  physicians  can  then  concentrate  their 
attention  on  cases  with  less  than  optimal  outcome.     Physicians  who  initially 
provided  the  care  reportedly  were  receptive  to  the  review  findings,  but  cur- 
rent experience  is  insufficient  to  determine  whether  the  quality  of  care  im- 
proves, once  this  information  is  made  available  to  them.  [11] 


METHODS  FOR  CHANGING  BEHAVIOR 

Quality  assurance  programs  are  generally  based  on  the  assumption  that  when 
inappropriate  care  is  detected,   specific  actions  can  be  taken  to  correct  the 
deficiency  and  ultimately  improve  the  quality  of  care.     However,  administrators 
of  almost  all  programs  visited  noted  the  extreme  difficulty  of  changing  provider 
behavior  after  deficiencies  are  identified.     In  a  few  instances,  infractions 
were  so  severe  as  to  warrant  a  report  to  the  state  board  of  medical  examiners; 
board  action  was  not  discernible,  however. 

Most  programs  rely  on  educational  means  for  encouraging  improvement,  but  the 
frequency  and  structure  of  such  activities  vary.  Bethesda-Lutheran  Hospital 
regularly  reports  the  results  of  medical  audits  or  conducts  appropriate 
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educational  sessions  at  weekly  medical  staff  meetings,  attended  by  about  25 
percent  of  the  staff.     The  findings  of  MCEs  at  Overlook  Hospital  are  considered 
by  the  apppropriate  department  of  the  medical  staff  and  eventually  are  reported 
to  the  board  of  trustees  to  determine  whether  further  action  is  warranted.  Re- 
sults of  review  by  the  San  Joaquin  Foundation  may  occasionally  lead  to  educa- 
tion activities  sponsored  by  the  medical  society  or,   in  extreme  cases,  refer- 
ral  to  state  authorities  for  appropriate  action. 

Several  approaches  have  been  tried  in  Utah  ranging  from  formal  education 
programs  and  specially  designed  hospital  rounds   to  a  series  of  increasingly 
more  severe  letters  that  discuss  inappropriate  care  uncovered  by  the  PACE 
review.     If  physicians  continue  to  provide  inappropriate  care  despite  earlier 
notices   from  the  PACE  program,  they  are  informed  that  their  claims  will  no 
longer  be  certified  for  payment.     The  decision  to  impose  payment  sanctions 
is  relatively  recent.  [12] 

Most  program  administrators  reported  that  physicians  who  consistently  differ 
from  review  guidelines  may  be  counseled  by  their  peers — in  the  case  of  a  hos- 
pital, this  would  generally  be  the  department  chairman.     Similarly,  most  pro- 
grams include  provisions  for  putting  a  physician  on  total  review,   in  which 
every  claim  or  hospitalization  by  that  physician  would  be  reviewed.  At 
Bethesda-Lutheran  Hospital,  documentation  of  inappropriate  care  is  recorded 
in  the  hospital's  file  on  the  responsible  physician.     However,  documentation 
of  compliance  is  also  recorded  in  the  physician's  file  if  the  deficiency 
has  been  corrected  upon  re-audit. 

The  Sacramento  Foundation  does  not  rely  on  an  educational  approach  and   is  quite 
explicit  in  applying  sanctions.     It  attempts  to  obtain  compliance  with  review 
guidelines  for  ambulatory  care  by  denying  reimbursement.     In  cases  where  care 
appears  to  be  inappropriate,  the  foundation  may  arrange  to  have  another  physi- 
cian examine  the  patient.     The  attending  physician  must  inform  the  patient  that 
there  will  be  a  consultation. 

A  few  programs  provide  for  patient  involvement.     In  Utah,  a  brochure  was  devel- 
oped to  explain  the  purpose  of  UPRO  and  the  review  program.     The  brochure  was 
intended  to  be  given  to  patients  at  the  time  of  admission,  but  apparently  some 
hospital  administrators  were  reluctant   to  distribute  it.     At  Multnomah,  a  letter 
of  denial  of  continued  stay  is  sent  first  to  the  patient,  although  the  attending 
physician  is  notified  that  the  letter  is  being  sent  and  presumably  may  discuss 
it  with  the  patient  in  advance.     Similarly,  the  San  Joaquin  Foundation  routinely 
notifies  the  patient  of  the  disposition  of  each  claim.     On  occasion,  patients 
have  appealed  the  foundation's  review  decision.     However,  the  extent  to  which 
this  makes  the  patient  more  aware  of  the  need  for  appropriate  utilization  is 
not  known. 

The  Texas  Department  of  Public  Welfare  mails  Medicaid  recipients  a  monthly 
packet  which  includes  the  Medicaid  card,  a  statement  of  all  services   for  which 
the  state  has  been  charged  to  be  verified  by  the  recipient,  and  any  special 
notices  such  as  the  availability  of  free  health  examinations  for  people  under 
21  years  of  age.     The  department  recently  instituted  a  program  whereby  monthly 
patient  profiles  will  identify  patients  who  appear  to  be  using  services 
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inappropriately.     For  those  people,   the  monthly  letter  will  not   include  the 
Medicaid  card.     Instead,   the  recipient  will  be  requested  to  attend  an  educa- 
tional  session  to  discuss  his  health  care.     The  Medicaid  card  will  be  provided 
during  the  session,  adding  an  incentive  to  attend.     The   impact  of  the  recipient 
education  program  on  subsequent  utilization  of  services  will  be  assessed. 

In  addition,  the  Texas  Medicaid  program  provides  each  recipient  with  a  medical 
assistance  record  book  in  which  all   services  are  supposed   to  be  recorded  by  all 
providers.     To  date,  about   50  percent  of  physicians  regularly  complete   it.  The 
record  permits  each  provider  to  review  the  care  being  rendered  by  all  others 
and  has  been  especially  useful   in  detecting  drug  incompatibilities. 

One  change  resulting  from  medical  audits  at  Bethesda-Lutheran  Hospital  was  the 
initiation  of  a  pre-surgical  unit,  which  reduces  the  number  of  hospital  days 
needed   to  prepare  a  patient   for  surgery.     Patient   satisfaction  with  the  unit 
was  ascertained  during  outpatient  follow-up.     A  pilot  program  to  explore  patient 
expectations  and  responsibilities   for  follow-up  care  is  planned.     The  extent  to 
which  it   improves  patient  outcome  will  be  assessed. 


METHODS  FOR  SELF -ASSESSMENT 


Specific   inquiries  were  made  in  each  program  visited  to  determine  whether  a 
formal  mechanism  for  self-assessment  existed.     The  Overlook  Hospital  review 
program  is  viewed  as  a  management  tool  which  includes  self-assessment.  Plans 
for  evaluation  through  medical  audit  have  been  included   in  the  development  of 
new  patient  care  programs,   such  as  cardiac  catheterization  and  outpatient 
abortions.     For  those  two  procedures,  audits  have  been  expanded  and  now  con- 
stitute a  routine  monitoring  mechanism.     Usually,  however,  programs  do  not 
have   formal  assessment  methods,   although  an  implicit,  continuing  process  of 
analysis  and  program  development  obviously  has  guided  the  evolution  of  such 
programs  as  the  Utah  Professional  Review  Organization  and  Bethesda-Lutheran 
Hospital's  review.     Some  programs  have  already  submitted  to  evaluation  by 
outside  groups,   and  relevant   studies  are  cited  in  the  next  chapter.     In  addi- 
tion, the  monitoring  of  hospital  reviews  by  PSRO  staffs  may  be  viewed  as  a 
form  of  self-assessment. 

The  Bethesda-Lutheran  staff  conducted  a  special  project  to  determine  whether 
their  review  process  had  adversely  affected  patient  outcome,   since  approxi- 
mately 65  percent  of  their  patients  are  discharged  before  the  50th  percentile 
of  CPHA  regional  norms  for  length  of  stay.     A  sample  of  re-admitted  patients 
was  studied  to  determine  whether  the  reasons   for  re-admission  could  be  re- 
lated to  the  prior  discharge.     They  concluded  that  no  re-admissions  were 
attributable  to  early  discharge  stimulated  by  the  review  process. 

Staffs  of  review  programs  were  asked  how  the  effectiveness  of  their  programs 
should  be  assessed.     Staff  of  the  Foundation  for  Health  Care  Evaluation  felt 
that   in  time  their  program  should  have  an  impact  on  the  health  status  of  the 
population  reviewed.     In  Colorado,  program  officials  felt  that  evaluation 
should  be  based  on  the  extent   to  which  they  succeeded   in  instituting  a  process 
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for  review.     The  Multnomah  staff  suggested  that  their  review  activities  should 
be  judged  on  reliability  and  validity.     The  executive  director  of  the  National 
Capital  Medical  Foundation  suggested  that  one  measure  of  effectiveness  might 
be  an  increased  availability  of  nursing  home  beds  as  a  result  of  the  ability 
of  concurrent  review  to  document  the  need  for  long-term  care.     In  other  areas, 
staff  were  ambivalent  and  felt  that  they  should  not  be  expected  to  evaluate 
the  effectiveness  of  their  review  programs. 

Despite  the  willingness  of  some  staff  to  discuss  measures  of  effectiveness, 
none  had  established  measurable  objectives  for  improving  the  quality  of  health 
care.     Some  had  delineated  activities  and  time  tables  for  implementing  review. 
Others  might  specify  the  number  of  hospitals  or  patients  that  should  be  covered 
by  review  by  a  certain  time,  or  the  number  of  MCEs  which  must  be  done  per  year. 
But  the  desired  accomplishments  with  respect  to  health  status  were  not  speci- 
fied. 


SUMMARY 

Most  quality  assurance  programs  reviewed  have  benefited  from  considerable 
financial  support  and  relatively  lengthy  experience.     They  are  generally 
viewed  as  some  of  the  "better"  quality  assurance  programs  and  are  not  a 
representative  national  sample.     Many  were  pioneers  in  the   field  of  quality 
assurance  and  serve  as  models  which  have  been  adapted  elsewhere.     Thus,  their 
contributions  are  real.     However,   they  have  also  encountered  some  difficulties 
which  may  be  equally  instructive.     In  drawing  conclusions   from  the  experiences 
of  these  programs,  one  runs  the  risk  of  prematurely  passing  judgment  about  a 
very  complex  problem  and  programs  that  may  not  have  had  sufficient  time  to 
prove  themselves.     Nevertheless,   the  information  should  be  useful   in  consider- 
ing the  need  for  mid-course  re-directions  which  may  be  ultimately  beneficial. 

Most  statements  of  program  goals  refer  to  the  need  to  ensure  high  quality 
medical  care  at  reasonable  costs,  but  do  not  include  measurable  objectives 
or  methods   for  improving  the  quality  of  care.     Indeed,   the  margin  by  which 
quality  might  be  improved  is  not  known.     No  program  official  was  able  to 
offer  even  a  rough  estimate  of  the  extent  of  deficient  or  inappropriate  care. 
Without  such  a  measure,   it  becomes  difficult  to  determine  whether  a  program 
is  achieving  its  objectives  and  whether  the  resultant  improvement  is  suffi- 
cient to  justify  program  expenditures. 

Most  programs  are  oriented  toward  users  of  institutional  or  ambulatory  health 
services,  rather  than  non-users,  and  with  a  few  exceptions,  do  not  address 
questions  of  access  to  care  and  under-utilization  of  services. 

Most  hospital  review  programs  place  primary  emphasis  on  concurrent  review 
activities  and  medical  care  evaluations,  as  required  by  PSROs ,  but  the  two 
activities  are  seldom  integrated.     There  is  considerable  variation  among 
programs  in  the  timing,  depth,  and  frequency  of  review.     Similarly,   the  ex- 
tent to  which  review  coordinators  and  physician  advisors  are  trained  and 
supervised  varies  from  program  to  program.     Common  problems  have  been  identi- 
fied in  conducting  medical  care  evaluations:     the  incompatibility  between  JCAH 


52 


and  PSRO  requirements;  the  difficulty  of  selecting  audit   topics  which  result 
in  the   identification  of  significant  problems  so  that  improvements  can  be 
made;  the  difficulty  of  developing  criteria  relevant   to  all  patients  without 
becoming  too  general;  and  the  difficulty  of  achieving  change,  once  deficien- 
cies have  been  identified.     Innovative  efforts  which  address  these  problems 
are  underway. 

Some  ambulatory  care  quality  assurance  programs  are  evolving  as  alternatives 
to  claims  review.     They  have  the  potential  of  permitting  greater  emphasis  on 
quality  by  viewing  the  provision  of  medical  care  over  time  (rather  than  indi- 
vidual encounters)  and  assessing  access  and  health  outcomes.     However,  they 
require  further  refinement  and  evaluation  before  being  widely  implemented. 

Most  programs  rely  on  educational  methods   for  encouraging  improvements  in 
performance.     Although  there  are  some  internal  appraisals  of  the  effects  of 
review,  no  program  had  established  a  formal  mechanism  for  self-assessment. 
Some  have  submitted  to  independent  evaluations  by  external  groups,  however. 

STEERING  COMMITTEE  RECOMMENDATIONS 

1.  Within  quality  assurance  programs,  the  review  components  should  be  better 
integrated.     Further  development  of  profile  analysis  may  help  to  identify 
quality  deficits  and  permit  better  targeting  of  both  concurrent  review  and 
MCEs ,  but  a  conscious  effort  is  needed  to  link  the  three  types  of  review. 

The  policy  of  delegating  responsibility  to  hospitals  for  conducting  MCEs 
independently  of  concurrent  review  should  be  reconsidered,   since   it  may 
encourage  fragmentation.     Within  hospitals,  PSRO  review  activities  should 
be  better  integrated  with  prior  utilization  review  and  other  quality  assur- 
ance activities. 

2.  There  should  be  fewer,  better  designed,  and  better  evaluated  MCEs,  and 
experimentation  should  be  encouraged.     The  JCAH  and  PSRO  requirements  should 
be  compatible  in  content,  as  well  as  in  their  numerical  requirements.  Hospi- 
tals should  be  permitted  to  count  re-audits  of  completed  audits  in  fulfilling 
numerical  requirements. 

3.  Establishment  of  quality  criteria  would  be  facilitated  by  increased  know- 
ledge of  the  efficacy  of  medical  procedures  derived  from  clinical  research. 
The  Department  of  Health,  Education,  and  Welfare  should  further  specify  the 
responsibilities  of  its  component  agencies  in  this  area  and  increase  available 
fund  s . 

4.  Quality  assurance  programs  should  further  specify  their  objectives  for 
improving  health  status  and  establish  internal  self-assessment  units  for 
program  evaluation.     The  Department  of  Health,  Education,  and  Welfare  should 
provide  technical  assistance  for  such  activities.     Appropriate  linkage  with 
health  services  research  centers  should  be  encouraged. 
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5.  The  components  of  quality  currently  addressed  by  quality  assurance  programs 
should  be  expanded.     Access   to  care  and  potential  under-utilization  should  be 
assessed.     This  requires  attention  to  the  availability  of  health  care  pro- 
viders and  facilities,  appropriate  links  between  levels  of  care,  and  policies 
(such  as  on-call  arrangements)  to  assure  that  services  are  easily  accessible 

to  patients  who  need  care.     Better  working  relationships  and  exchange  of  infor- 
mation should  be  encouraged  between  quality  assurance  programs  and  health 
planning  agencies.     Benefit  packages  and  reimbursement  policies  should  be  ex- 
amined to  determine  their  impact  on  the  quality  of  care,  particularly  with  re- 
spect  to  coverage  of  long-term  care  (both  institutional  and  non-institutional). 

6.  A  priority  for  research  is  the  systematic  accumulation  of  data  to  des- 
cribe current  patterns  of  care,  determine  the  reasons  for  variation,  and 
identify  deficiencies.     This  information  would  provide  a  better  estimate 
of  the  margin  by  which  quality  and  utilization  might  be  improved,  which  in 
turn  would  help  to  determine  the  magnitude  of  the  required  quality  assurance 
effort . 


54 


FOOTNOTES 


Chapter  3 


1 

U.S.   Department  of  Health,  Education,   and  Welfare,  National  Center  for 
Health  Services  Research  and  Development,  Experimental  Medical  Care 
Review  Organization  (EMCRO)  Programs,  Arthur  D.  Little,  Inc., 
DHEW  Pub.  No.   (HSM)  13-3017  (1973). 

2 

Michael  Goran  et  al . ,  "The  PSRO  Hospital  Review  System,"  Medical  Care 
Supplement  13  (April   1975):  15. 

3 

Ibid .  ,   p.  17. 
4 

Leonard  Rubin,  Comprehensive  Quality  Assurance  System:     The  Kaiser-Permanente 
Approach  (Alexandria,  Va . :  American  Group  Practice  Association,  1975) . 

5 

Charles  Lewis,  EMCRO  Annual  Report  (Los  Angeles:     UCLA  Center  for  the  Health 
Sciences,   1975).     See  also  Sheldon  Greenfield  et  al.,  "Peer  Review  by 
Criteria  Mapping:     Criteria  for  Diabetes  Mellitus,"  Annals  of  Internal  Medicine 
83  (December  1975):  761-70. 

6 

Sheldon  Greenfield  et  al . ,   "Protocol  Management  of  Dysuria,  Urinary  Frequency, 
and  Vaginal  Discharge,"  Annals  of  Internal  Medicine  81  (October  1974):  452-57; 
Greenfield  et  al . ,   "Management  of  Low  Back  Pain  by  a  Protocol  and  a  Physician 
Extender,"  Western  Journal  of  Medicine  123  (1975);   and  Anthony  Komaroff  et  al., 
"Protocols  for  Physician  Assistants:     Management  of  Diabetes  and  Hypertension," 
New  England  Journal  of  Medicine  290  (7  February  1974):  307-12. 

7 

Department  of  Health,  Education,   and  Welfare,  Washington,  D.   C. ,  Minutes  of 
Meeting  of  the  National  Professional  Standards  Review  Council,   12  July  1976. 


55 


8 

Robert  Brook  and  Kathleen  Williams,  "Evaluation  of  the  New  Mexico  Peer 
Review  System  1971-73,"  Medical  Care,  forthcoming. 

9 

Rubin,  Comprehensive  Quality  Assurance  System. 
10 

Alvin  Mushlin  et  al . ,  "The  Use  of  Patient-Reported  Problem  Status  in 
Evaluating  Ambulatory  Care,"  presented  at  the  National  Meeting  of  the  American 
Federation  for  Clinical  Research,  Atlantic  City,  N.  J.,   3  May  1975. 

11 

Alvin  Muchlin,  Francis  Appel,  and  Daniel  Barr ,  "Quality  Assurance  in 
Ambulatory  Care:     A  Strategy  Based  on  Outcome  Assessment,"  1976. 
(Mimeographed . ) 

12 

James  Cannon,  PACE  project  manager,  personal  letter,  28  June  1976. 


Chapter  4 


EFFECTIVENESS  OF  QUALITY  ASSURANCE  PROGRAMS 


A  wide  range  of  material  was  gathered  to  show  the  initial  effects  of  quality 
assurance  programs,   including  impressions  and  anecdotes,   as  well  as  more  sys- 
tematically gathered  evaluative  data.     Because  documentation  is  frequently 
inadequate,  a  definitive  assessment  of  effectiveness  is  not  possible  at  this 
time.     Nevertheless,   the  information  is  sufficient  to  permit  a  preliminary 
assessment  of  current  programs  and  to  suggest  areas  in  which  additional  re- 
search and  evaluation  are  required. 

General  problems  in  evaluating  program  effectiveness  and  limitations  of  cur- 
rent measures  are  discussed  before  presenting  information  about  the  effective- 
ness of  the  programs  visited.     All  issues  are  discussed  in  more  detail  in  the 
forthcoming  priority  paper  on  "Cost  and  Cost  Effectiveness  of  Health  Care 
Quality  Assurance  Programs." 


GENERAL  PROBLEMS  IN  EVALUATION 

Few  quality  assurance  programs  were  instituted  with  explicit  objectives  and 
plans  for  program  evaluation.     Inferences  were  made  by  the  study  staff  about 
the  relative  emphasis  given  to  cost  and  non-cost  components  of  quality  re- 
view.    The  translation  of  general  program  goals  into  policies  and  procedures 
for  routine  operations  was  examined  to  shed  further  light  on  specific  program 
objectives  and  orientations.     Nevertheless,   the  absence  of  clearly  defined  ob- 
jectives complicates  the  assessment  of  program  effectiveness. 

Furthermore,   in  most  quality  assurance  programs,   and  particularly  in  an  un- 
dertaking as  complex  as  the  national  PSRO  program,   there  are  many  different, 
sometimes  conflicting,  objectives.     The  declaration  of  purpose  in  the  PSRO 
legislation  refers  to  the  "effective,  efficient,   and  economical  delivery  of 
health  care  services  of  proper  quality...."   [1]     In  fulfilling  this  purpose, 
many  specific  objectives  and  required  activities  are  determined  by  the  Bureau 
of  Quality  Assurance.     Members  of  local  PSROs  may  place  additional  or  greater 
emphasis  on  the  preservation  of  local  autonomy  and  may  concentrate  their  ef- 
forts initially  on  specific  tasks  required  to  establish  an  independent  program 
and  begin  routine  review  activities.     The  demands  of  program  implementation  may 
overshadov.  the  broader  goal  of  improving  the  quality  of  health  care,   so  that  an 
appropriate  evaluative  measure  for  the  short-term  might  be  the  extent  to  which 
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tasks  associated  with  program  implementation  and  operation  are  completed. 
However,  most  programs  reviewed  in  this  study  have  passed  the  implementa- 
tion stage.     Furthermore,   the  study  addresses  broader,   long-term  issues  of 
quality  assurance.     Measures  of  effectiveness,   therefore,  must  also  be  viewed 
from  a  broader  perspective. 

Throughout  this  report,   the  primary  objective  of  quality  assurance  is  as- 
sumed to  be  maintaining  or  improving  the  quality  of  health  care.  Program 
effectiveness   is  regarded  as  the  extent  to  which  this  objective  is  achieved 
and  should  be  measured  by  improvements  in  health  status  or  patient  satisfac- 
tion (non-cost  components  of  quality)  and  the  conservation  of  resources  (cost 
components  of  quality).     Determination  of  cost  effectiveness  involves  the  re- 
lationship between  the  cost  of  operating  the  review  program  and  effectiveness 
as  measured  by  improvements  in  health  status  and  satisfaction  and  reduction  in 
the  cost  of  health  care.     If  the  quality  assurance  program  is  an  addition  to  a 
health  care  delivery  system,   the  cost  of  quality  assurance  is  the  difference 
between  the  cost  of  the  delivery  system  with  and  without  quality  assurance 
activities . 

Even  if  objectives  are  well  specified,  differences  among  programs  limit  com- 
parisons of  their  effectiveness.     Differences  in  the  type  of  medical  care  re- 
viewed and  approaches  to  review  make  it  difficult  to  compare  review  costs  or 
effectiveness.     Assessments  of  the  relative  effects  of  different  types  of  re- 
view mechanisms  might  be  attempted  by  comparing  programs  according  to  the 
magnitude  of  changes  within  them  and  assuming  all  other  influences  could  be 
held  constant.     However,   adequate  baseline  data  are  not  available  to  permit 
bef ore-and-af ter  measurements. 

Since  is  is  not  known  when  the  effects  of  a  review  system  should  become  ap- 
parent, it  is  difficult  to  determine  when  measurements  should  be  made.  One 
may  argue  that  the  anticipation  of  quality  assessment  and  the  heightened 
awareness  of  health  care  providers  to  increasing  public  scrutiny  may  change 
performance  patterns  even  in  the  absence  of  a  formal  review  system.  Nelson 
has  noted  that  the  act  of  establishing  process  criteria  for  psychosis 
resulted  in  increased  compliance  with  criteria  from  64  to  88  percent  (based 
on  50  hospital  stays)  before  the  assessment  was  formally  initiated.  [2] 

On  the  other  hand,   it  may  be  unrealistic  to  expect   immediate  changes.     If  a 
program  is  expanding  its  base  of  support  from  physicians  and  hospitals  at  the 
same  time  that   it  becomes  operational,  program  administrators  may  be  reluc- 
tant to  impose  stringent  criteria  until  all  providers  have  been  brought  within 
the  system.     Once  change  is  documented,   it  is  important  to  determine  whether  it 
is  permanent.     Although  an  initial  impact  resulted  from  the  imposition  of 
the  Approval  by  Individual  Diagnosis  (AID)  program  of  recertif ication  for 
continued  hospital  stay  in  New  Jersey,  Bailey  and  Riedel  have  shown  that 
the  effect  was  short-lived.    [3]     After  physicians  became  more  familiar  with 
the  program,   they  apparently  returned  to  their  prior  utilization  practices. 

Even  if  an  appropriate  evaluative  design  can  be  developed,   the  identifica- 
tion of  variables  to  indicate  effectiveness  and  specification  of  reliable 
measures  present  additional  difficulties. 
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MEASUREMENT  PROBLEMS 


The  effect  of  review  on  quality  of  care  independent  from  utilization  and 
cost  is  the  most  difficult  to  assess.     Health  status  of  a  population  is 
an  obvious  referent  for  assessing  the  outcome  of  health  care.     It  is  typi- 
cally measured  by  mortality  and  morbidity  statistics.     However,  a  sub- 
stantial lag  time  can  be  expected  before  changes  in  health  status  are 
reflected  in  statistics.     Casual  interpretation  is  complex  because  of  the 
multiple  factors  influencing  health  and  the  limited  effect  of  medical  care. 
Another  end  result  index  is  patient  satisfaction,  but  consistent  measures 
are  not  currently  in  use,  and  there  are  limited  plans  for  the  collection 
of  such  data. 


Some  programs  review  intermediate  or  more  proximate  outcomes,  which  may 
include  the  incidence  of  preventable  complications  or  the  performance  of 
critical  therapeutic  procedures  as  outcomes  of  the  diagnostic  decision- 
making process.     Such  assessments  are  often  included  in  medical  care  eval- 
uation studies.     Unfortunately,  there  have  been  few  attempts  to  accumulate 
information  on  both  successful  and  unsuccessful  MCEs ,  assign  values  to  the 
data,  and  aggregate  them  in  a  manner  which  would  permit  an  overall  assess- 
ment of  effectiveness.     Methods  for  doing  so  are  not  well  developed.  The 
number  of  MCEs  performed  under  PSROs  will  be  reported  by  the  PSRO  informa- 
tion system,  but  this  has  limited  utility  for  evaluation. 


Improved  methods  are  needed  to  identify  and  aggregate  the  effects  on  health 
status  which  result  from  health  care.     It  may  not  be  necessary  to  express 
those  improvements  in  dollars  or  comparable  units  of  resource  expenditures. 
But  better  summary  measures  must  be  devised  in  order  to  permit  assessment  of 
improvements  which  stem  from  quality  assurance  programs,  continuing  medical 
education,  and  other  activities  designed  to  elevate  the  health  status  of  the 
population . 


Because  of  the  close  link  between  quality  and  utilization  of  services,  the 
relative  ease  of  measurement,  and  the  associated  emphasis  on  cost  control, 
impact  is  often  measured  in  terms  of  utilization,  as  expressed  in  costs. 
Customary  measures  and  their  limitations  are  discussed  below. 


Admissions  and  admission  denials.     Data  on  admission  rates  and  admission 
denials  are  not  uniformly  collected  and  are  easy  to  misinterpret.  The 
population  base  for  calculating  rates  may  be  difficult  to  determine.  A 
denied  elective  admission  may  become  a  necessary  or  emergency  admission. 
Alternatively,  the  absence  of  admission  denials  does  not  mean  that  the  system 
is  not  effectively  deterring  admissions,   since  physicians  may  not  attempt  to 
admit  cases  they  suspect  will  be  denied. 


Denied  requests  for  continued  stay.     A  denied  request  for  continued  hospital 
stay  is  also  difficult  to  interpret,  since  an  inappropriate  denial  may  result 
in  a  subsequent  need  for  additional,  possibly  more   intensive,  care.     Also,  the 
possibility  of  denial  may  result  in  patients  being  discharged  earlier  than 
would  otherwise  be  the  case,  so  the  opportunity  for  denial  is  eliminated. 
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Reductions  in  length  of  stay.     Average  length  of  stay  before  a  quality  assur- 
ance program  is  initiated  helps  define  the  maximum  savings  that  concurrent 
review  can  achieve.     One  may  assume  that  the  longer  the  average  length  of 
stay,  the  greater  the  probability  of  finding  unnecessary  days  of  stay,  but 
this  may  not  be  true.     Referral  centers  may  have  more  difficult  cases  which 
require  longer  stays,  and  other  social  and  environmental  conditions  may  af- 
fect medical  decisions.     At  the  other  extreme,  however,  one  can  not  expect 
to  achieve  major  savings  if  length  of  stay  is  already  low. 

To  the  extent  that  admissions  are  denied  or  deterred  by  the  review  process, 
cases  admitted  are  likely  to  be  more  serious  and  may  require  longer  stays. 
In  some  hospitals,  therefore,  an  effective  review  program  may  leave  unchanged 
or  even  increase  length  of  stay.     Finally,  average  length  of  stay  data  reflect 
a  combination  of  case,  provider,  and  service  mix  for  any  given  facility,  and 
for  a  combination  of  facilities  whenever  an  areawide  average  is  used.  There- 
fore, data  should  be  adjusted  for  age,  sex,  and  diagnosis;  and  the  potential 
influences  of  provider  and  service  mix  should  be  considered  when  interpreting 
the  data.     Available  data  are  seldom  adjusted,  however. 

Bed  days  saved.     Estimates  of  bed  days  saved  per  1,000  population  are  some- 
times used  as  a  means  of  incorporating  the  deterrent  effect  of  both  denied 
admissions  and  extended  stay  requests.     These  estimates  reflect  the  number 
of  bed  days  which  would  have  been  used  in  the  absence  of  the  review  programs, 
based  on  one  of  two  techniques.     One  requires  an  extrapolation  of  demand  for 
bed  days  before  establishment  of  the  review  program,  based  on  several  years 
and  on  relatively  large,  stable  populations  to  minimize  year-to-year  varia- 
tions.    This  method  is  complicated  by  the  difficulty  of  defining  service 
areas  and  the  potential  for  changes  in  available  facilities.     The  extrapola- 
tion should  be  accompanied  by  an  analysis  of  the  reasons  for  trends  observed, 
which  may  not  be  feasible  after  the  fact.     This  technique  requires  further 
refinement  before  being  widely  applied. 

The  second  method  is  a  comparison  of  an  area  with  a  review  program  and  another 
area  with  no  review  program  or  a  different  approach  to  review.  This  technique 
assumes  that  by  design  or  by  the  accident  of  a  "natural"  control,  the  two  pop- 
ulations are  under  similar  influences  except  for  the  review  programs,  and  that 
for  analytic  purposes  there  will  be  no  interaction  between  the  two  populations. 
It  has  not  been  used  often,  but  may  receive  more  attention  if  there  continue  to 
be  some  areas  without  PSROs .     More  adequate  trend  data  are  needed,  however. 

Cost  of  alternate  care.     If  bed  days  are  saved,   patients  who  initially  were 
considered  for  acute  hospital  care  may  need  care  in  a  different  setting.  There 
will  be  costs  for  this  other  treatment.     Therefore,  the  total  value  of  bed  days 
saved  does  not  equal  the  value  of  per  diem  times  number  of  days  saved,  but 
should  be  adjusted  downward  to  reflect  the  estimated  cost  of  alternative  care. 

Riedel  et  al .  analyzed  hospitalization  data  for  federal  employees  in  Washington, 
D.  C.  who  were  insured  by  Group  Health  Association  (a  prepaid  health  plan)  or 
high  option  Blue  Cross-Blue  Shield.     They  found  that  GHA  enrollees  used  40  per- 
cent  fewer  bed  days  than  the  population  served  by  fee-for-service  physicians; 
however,  the  total  cost  of  care  for  the  two  populations  was  about  the  same. 
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Additional  costs   for  GHA  ambulatory  care  offset  hospital  cost  savings.  [4] 
A  similar  conclusion  was  reached  by  Perkoff  et  al .  who  studied  an  experi- 
mental prepaid  group  practice  at  Washington  University.     Savings  resulting 
from  a  23  percent  reduction  in  hospital  days  did  not  compensate  for  the  in- 
creased primary  care  and  specialty  ambulatory  care  costs.   [5]     To  fully  ex- 
plain these  findings,  a  more  detailed  analysis  is  needed  of  the  units  of 
service  used,  cost  per  unit,  and  influence  of  the  benefit  package.  Never- 
theless, if  areawide  hospitalization  criteria  were  based  on  these  prepaid 
group  practices,  the  net  reduction  in  hospital  bed  days  would  not  result 
in  an  equivalent  reduction  in  health  care  expenditures. 

Variable  expenses.     The  use  of  average  cost  per  day  of  hospitalization  as 
the  value  of  a  bed  day  saved  implies  that  all  expenses  vary,  so  that  the 
reduction  in  total  costs  will  be  proportionate  to  the  reduction  of  bed  days. 
However,  all  costs  do  not  vary.     Some  costs  are  fixed — at  least  for  the  short 
run.     Cost-plus  methods  for  reimbursement  provide  few  incentives  for  hospital 
administrators  to  manage  aggressively  all  the  costs  that  might  vary  with  oc- 
cupancy rates.     Moreover,  days  saved  are  likely  to  be  days  of  less  intensive 
care  and,  therefore,  less  expensive  care.     Thus,  full  per  diem  evaluation  of 
bed  days  saved  overstates  true  hospital  savings. 

The  proportion  of  hospital  costs  which  vary  according  to  occupancy  is  not 
known  and  needs  further  study.     The  Cost  of  Living  Council  based  Phase  IV 
hospital  regulations  on  the  assumption  that  40  percent  of  average  costs  were 
variable,  but  noted  that  with  a  declining  volume  of  admissions,  the  percent 
of  variable  costs  might  be  less.   [6]     The  relationship  between  utilization 
review  activities  and  variable  costs  has  not  been  examined  in  detail.  Never- 
theless, the  percent  of  variable  costs  is  unlikely  to  be  constant  over  a  wide 
range  of  occupancy  rates.     The  ability  to  control  variable  costs  may  further 
depend  on  the  period  of  time  over  which  the  reduction  in  occupancy  rates 
occurs,  the  level  of  occupancy  rates  before  length  of  stay  is  reduced,  and 
the  influence  of  the  area  planning  and  rate  regulating  authorities  in  con- 
trolling availability  and  utilization  of  hospital  resources. 


If  the  concepts  of  variable  costs  and  costs  of  alternate  care  are  incor- 
porated into  the  estimate  of  the  value  of  bed  days  saved,  the  result  may 
help  to  identify  the  amount  of  savings  needed  to  achieve  the  threshold  of 
cost  effectiveness. 


Communitywide  effects.     Estimates  of  cost  containment  based  on  areawide 
statistics  introduce  an  additional  factor  for  consideration.     Area  savings 
are  not  necessarily  the  sum  of  savings  of  individual  institutions,  but  are 
affected  by  the  relative  variability  of  hospital  expenses  in  different  facil- 
ities, the  amount  of  revenues  exceeding  costs  (as  a  measure  of  economic  pres- 
sure to  find  economies  in  operations),  and  the  ability  of  each  facility  in  an 
area  to  attract  additional  physicians.     Hospitals  with  higher  investments,  ad 
vanced  technology,  and  greater  prestige  may  try  to  attract  additional  physi- 
cians and  patients  as  shorter  lengths  of  stay  threaten  to  reduce  their  rev- 
enues,  instead  of  closing  wards  and  sections.     Closing  wards  might  further  in 
crease  their  per  diem  rates,  reduce  their  competetive  status,  and  create  othe 
problems.     Since  high  technology  hospitals  are  usually  facilities  with  higher 
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per  diem  rates,   a  shift  of  patient  population  from  Lower  to  higher  cost 
facilities  could  offset  savings  derived  from  a  reduction  in  bed  days  used. 

Ambulatory  impact  measures.     Effects  of  ambulatory  quality  assurance  programs 
are  usually  determined  by  the  numbers  and  types  of  services  rendered  as 
stated  on  claims  forms.     Forms  are  examined  for  appropriateness  of  care 
rendered  and  the  appropriateness  of  the  charge,   usually  based  on  the  designated 
unit  of  a  relative  value  scale  (RVS).     Determination  of  the  appropriateness 
of  care  requires  medical  or  peer  judgment.     The  appropriateness  of  charges, 
however,   is  an  administrative  decision  that   is  made  by  claims  review  programs 
which  do  not   include  an  assessment  of  quality.     In  assessing  the  effectiveness 
of  ambulatory  review  activities,   the  reasons  for  which  judgments  are  made 
should  be  identified,   since  they  require  different  personnel  with  differing 
levels  of  training  and  associated  costs.     As  with  hospital-based  programs, 
the  difficulty  of  assessing  the  deterrent  effect  of  review  and  the  importance 
of  the  influence  of  patient,   provider,   and  service  mix  and  the  cost  of  alter- 
nate care  should  be  noted. 

Cost  savings  and  shifting  of  costs.     A  denial  of  hospital  certification  or 
the  denial  or  reduction  of  ambulatory  claims  may  result   in  a  shifting  of 
costs,   rather  than  savings.     Although  a  retrospective  review  of  care  already 
provided  may  save  money  for  the  fiscal  intermediary,  often  the  patient,  the 
community,  or  another  intermediary  will  pay  the  provider  instead. 

Attention  to  unmet  health  needs.     The  ability  to  meet  previously  unmet  needs 
for  health  care  is  a  valid  measure  of  the  effectiveness  of  quality  assurance 
programs,   even  though  costs  and  utilization  of  services  would  increase. 
However,  most  existing  quality  assurance  programs  do  not  address  unmet  health 
needs . 

Measures  of  program  effectiveness  and  their  limitations  obviously  influence 
the  resulting  judgments.     The  PSRO  Program  Evaluation  Plan  is  noteworthy 
because  it  is  one  of  the  first  attempts  to  design  a  systematic  assessment 
of  the  impact  of  a  federal  program  before  nationwide  implementation,   and  its 
introductory  sections  reflect  an  appreciation  of  the  associated  complexi- 
ties.   [7]     Nevertheless,   the  method  selected  for  assessing  cost  savings  from 
hospital  review  makes  no  explicit  allowance  for  cost  of  alternate  care,  makes 
inadequate  allowance  for  fixed  and  variable  costs   in  estimates  of  "savings," 
and  makes  no  allowance  for  the  change  in  cost  of  care  that  would  result  from 
a  shift  between  lower  and  higher  cost  facilities.     Accordingly,  savings 
which  may  result  from  PSRO  review  will  be  overstated. 

EFFECTS  OF  REVIEW  ON  QUALITY 

The  difficulty  of  separating  improvements   in  quality  of  care  from  effects  on 

utilization  and  cost  and  making  an  overall  assessment  of  the  effectiveness 

of  review  activities  was  noted  above.     However,   the  kinds  of  quality  improve- 
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ment  said  to  derive  from  medical  care  evaluation  studies  can  be  described 
anecdotally.    [8]     Whether  the  improvements  are  causally  associated  with  the 
imposition  of  a  review  program  or  other  factors   is  a  matter  for  conjecture. 
To  a  limited  degree,   the  costs  of  conducting  MCEs  are  also  known. 

The  quality  assurance  program  at  Bethesda-Lutheran  Hospital  emphasizes 
medical  audit.     Audit  activities  were  intensified  simultaneously  with  a 
reorganization  of  the  hospital  administrative  and  committee  structure  and 
the  institution  of  a  problem-oriented  medical  record.     Examples  of  changes 
related  to  medical  audit,  which  are  thought  to  have  improved  the  quality  of 
care,  follow: 

.   purchase  of  new  equipment  including  central  fetal  monitor  and  portable 
cardiac  monitors  in  outpatient  department; 

.  creation  of  specific  patient  care  units,   such  as  a  presurgical  floor, 
post-myocardial  infarction  and  cardiac  floor,   and  a  proposed  post- 
surgical critical  care  floor; 

.   establishment  of  a  "Surgaday"  program  in  which  surgical  patients  are 
admitted  and  discharged  on  the  same  day; 

.   education  programs  including  a  general  staff  conference  and  tour  of 
the  occupational  therapy  department,  which  resulted  in  increased 
use  of  rehabilitative  services; 

.   increased  compliance  with  medical  audit  criteria  upon  re-audit; 

.   improved  documentation  in  the  medical  record; 

.  detection  of  previously  undetected  cases  of  hypertension  as  a  result 
of  an  audit  conducted  by  the  nursing  staff; 

.   institution  of  patient  education  programs;  and 

.  hiring  of  registered  nurses  to  meet  new  demands  resulting  from  the 
increased  number  of  patients  requiring  intensive  nursing  care  and  for 
telephone  follow-up  of  "Surgaday"  patients. 

Similar  information  has  been  presented  to  indicate  impact  of  the  CQAS  program 
as  implemented  in  the  Northern  California  region  of  Kaiser  Permanente: 

.   institution  of  new  systems  to  transfer  information  on  drug  allergies 
and  prescriptions  from  the  hospital  to  the  ambulatory  record,  to 
provide  information  on  current  drug  use  for  each  patient  visit,  and 
to  provide  clinical  information  to  the  anesthesiologist  before  surgery; 

.  contracting  with  a  new  vendor  for  improved  arm  bands  for  patient  identi- 
fication; 


earlier  hospital  discharge  after  establishing  policies  to  ensure  that 
patients  do  not  remain  hospitalized  for  suture  removal; 
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.  specific  improvements  in  administration  of  immune  globulin  to  post- 
partum patients,  use  of  blood  fractions,  hypertension  detection, 
preoperative  electrocardiograms,  pediatric  neonatal  appraisal; 

.  purchase  of  equipment,  such  as  more  baby  scales; 

.  reduced  waiting  time  for  radiological  contrast   studies;  and 

.   improvements   in  problems  associated  with  "no  show"  outpatient  appoint- 
ments. 

Overlook  Hospital  has  had  extensive  experience  with  medical  audits,  which 
are  thought   to  have  achieved  the  following: 

.  more  appropriate  use  of  packed  cells,  rather  than  whole  blood; 

.  decrease   in  inappropriate  postoperative  length  of  stay; 

.  decrease   in  inappropriate  retrograde  pyelography  performed  on  young 
pat  ients ; 

.  increased  use  of  rehabilitation  facilities  for  stroke  patients;  and 

.  98-100  percent  compliance  with  surgical  criteria  for  hysterectomies, 
caesarian  sections,  and  cholecystectomies  upon  initial  audit,  which 
was  maintained  upon  re-audit. 

A  review  of  patients  at  Overlook  Hospital  with  hip  fractures  showed  that 
over  30  percent  of  the  patients  had  decubitus  ulcers.     As  a  result,  a 
hospital-wide  monitoring  system  and  a  pressure  sore  prevention  and  treatment 
protocol  were  developed.     The  decubitus  ulcer  program  is  directed  by  a 
rehabilitation  nurse  who  was  hired  as  a  result  of  an  audit  in  1973  of  patients 
with  strokes. 

Medical  audits  at  Mt .   Sinai  Hospital   in  New  York  City  led  to  a  series  of 
department  conferences  and  reported  improvements  in  medical  record  documen- 
tation.    An  audit  of  patients  with  impacted  teeth  led  to  recommendations 
that  all  dental  patients  receive  a  physical  exam,  that  radiographic  findings 
be  documented,  and  that  all  morbidity  be  described   in  progress  notes  and 
the  discharge  summary.     As  a  result  of  an  audit  of  patients  with  myocardial 
infarction,  staff  initiated  a  log  to  record  requests  for  admission  to  the 
coronary  care  unit  and  the  disposition  of  such  cases,   and  an  intermediate 
care  area  was  created.     A  study  of  appendectomies  led  to  the  recommendation 
that  a  surgical  resident  be  available  24  hours  daily  for  emergency  room 
consultation  and  that  the  incidence  of  normal  appendices  removed  surgically 
should  not  exceed  10  to  15  percent. 

The  San  Joaquin  Foundation  reports  that  the   frequency  of  certain  procedures, 
such  as  cholecystectomies  and  tonsillectomies,  has  been  reduced  by  placing 
particular  physicians  on  total  review.     Computer  printouts  from  the  dis- 
continued Medi-Cal  review  indicate  that  the  dollar  amounts  of  claims  were 
reduced  and  the  number  of  pap  smears  and  appropriate  immunizations  increased. 
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An  independent  evaluation  of  the  Patterns  of  Treatment  program  confirms 
a  reduction  in  unnecessary  injections.  [9] 

Intervention  by  nurse  coordinators  at  the  San  Joaquin  PSRO  led  to  quality 
improvements  for  specific  patients.     A  review  of  an  EKG  report  which  arrived 
after  a  patient  had  been  discharged  led  to  the  patient's  re-hospitalization 
for  treatment  of  myocardial   infarction.     Repeat  hospital  admissions  of  an 
80-year  old  man  were  eliminated  after  arrangements  were  made  for  the  visiting 
nurse  to  supervise  his  medications  at  home.     A  patient  was  transferred  to 
a  lower  level  of  care  after  the  nurse  coordinator  convinced  a  nursing  home 
administrator  to  stock  certain  types  of  packing  and  dressings  required  for 
his  care. 

Although  the  Utah  experience  indicates  a  general  disappointment  with  tra- 
ditional audits,  MCEs  were  instrumental  in  changing  the  medical  school 
curriculum  for  physical  diagnosis.     Content  of  medical  grand  rounds  was 
sometimes  tailored  to  address  identified  deficiencies.     More  innovative 
MCEs  involving  the  achievement  of  management  objectives  for  hospitalized 
patients  and  potential   intervention  in  the  patient  care  process  are  being 
evaluated . 

Limited  information  on  the  cost  of  conducting  MCEs  in  individual  hospitals 
is  available.     The  differences  in  cost  have  not  been  thoroughly  analyzed 
and  may  be  affected  by  the  number  of  studies  performed,  the  subject  matter 
examined,  associated  personnel  costs,   and  the  efficiency  and  quality  of 
their  work.     The  cost  of  MCEs  per  admission  at  Bethesda-Lutheran  is  about 
$5  for  all  patients;  the  comparable  cost  at  Overlook  is  about  $4.  Most 
recent  information  on  MCEs  reported  to  the  Bureau  of  Quality  Assurance 
shows  a  cost  of  $.86  for  the  Colorado  PSRO  and  $1.83  for  the  Utah  PSRO. 
These  costs  are  incomplete,  however,  since  the  MCE  components  of  PSRO  re- 
view are  still  being  developed  and  most  costs  are  being  borne  by  the  hospi- 
tals and  are  not  included  in  the  PSRO  cost  data.     The  cost  of  the  Kaiser 
CQAS  system  is  about  $.25  per  member  per  year  for  both  ambulatory  and 
hospital  care.     The  operating  cost  of  the  Indian  Health  Services'  Health 
Information  System  on  which  quality  assessment  activities  are  based  is 
about  $12  per  person  per  year. 

The  changes  summarized  above  have  no  doubt  improved  the  technical  quality 
of  the  process  of  care  and  should  result  in  improved  health  outcomes,  even 
though  it  is  difficult  to  quantify  the  outcome  benefits  achieved — particular- 
ly for  such  activities  as  detecting  previously  unknown  cases  of  hypertension, 
assuming  treatment  is  successful.     Greater  uncertainty  arises,  however,  in 
attempting  to  determine  whether  the  improvements  are  sufficient   to  justify 
the  resources  expended.     The  total  magnitude  of  activities  (successful  and 
unsuccessful)  is  not  known.     It  is  not  possible   to  conclude  that  the  improve- 
ments resulted  solely  from  the  presence  of  quality  assurance  programs,  since 
many  other  factors  were  operating  simultaneously.     Furthermore,  one  might 
argue  that  many  of  the  changes  should  be  the  routine  responsibility  of  a 
competent  hospital  or  health  professional  and  should  not  depend  on  the  imposi- 
tion of  a  special  quality  review  program.     With  such  inadequate  information, 
it  is  impossible  to  base  an  assessment  of  value  or  effectiveness  on  anything 
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other  than  personal  opinion.     Bits  of  additional   information  may  be  gleaned 
from  other  sources,  however. 

Fleisher  et  al .  have  recently  reported  the  results  of  their  "Mandate 
Project"   [10]   in  which  the  bi-cycle  concept  of  medical  audit   [11]  was 
voluntarily  implemented  in  ten  hospitals  between  1970  and   1973.     The  Mandate 
Project  was  based  on  the  assumption  that  its  success  would  depend  on  the 
extent  to  which  participating  hospitals  officially  endorsed   it.     The  first 
phase  consisted  of  a  bi-cycle  workshop  attended  by  representatives  from 
each  hospital  (a  member  of  the  board  of  trustees,  an  administrator,  a  repre- 
sentative from  medical  records,  and  three  physicians,  preferably  including 
the  director  of  medical  education  or  someone  with  responsibility  for  ad- 
ministering the  project).     To  continue  in  the  study,  each  hospital  had  to 
provide  a  letter,  signed  by  the  chairman  of  its  board,  administrator,  and 
chief  or  president  of  the  medical  staff,   indicating  an  understanding  of  the 
project  and  an  institutional  commitment  to  implement  it.     The  second  phase, 
auditing  two  or  three  diseases  or  conditions,  was  completed  by  eight  of  the 
ten  hospitals.     The  third  phase  required  that  actions  to  correct  identified 
deficiences  be  planned  and  carried  out  and  that  a  re-audit  be  conducted 
one  year  later  to  determine  the  level  of  improvement.     Five  hospitals  com- 
pleted the  third  phase. 

Differences  among  hospitals  in  audit  topics,  criteria,  and  improvement 
programs  hamper  precise  comparisons  of  degrees  of  change.  Nevertheless, 
least  improvement  was  apparent  in  the  hospitals  which  did  not  conduct  the 
corrective  action  efforts.     Factors  related  to  improvements  included  the 
presence  of  a  nucleus  of  committed  physicians,   involvement  of  the  medical 
staff  in  setting  criteria  and  standards,  active  cooperation  of  medical 
records  personnel,  and  support  by  the  administration  and  board  of  trustees, 
as  well  as  medical  staff.     The  overall  impact  of  the  project  is  difficult 
to  determine.     No  cost  data  were  reported. 

In  1973  the  American  Hospital  Association  initiated  a  demonstration  in  16 
hospitals  of  its  Quality  Assurance  Program,  which  had  been  developed  to 
meet  JCAH  and  expected  PSRO  requirements   for  medical  audit  and  utilization 
review.     The  specific  purposes  of  the  demonstration  were:  "1)  to  demonstrate 
and  evaluate  the  Quality  Assurance  Program  (QAP),  2)  to  develop  educational 
materials  to  support  the  introduction  and  maintenance  of  the  QAP,  and  3)  to 
encourage  the  adoption  of  quality  assurance  programs  in  community  hospitals 
throughout  the  United  States."  [12] 

During  the  one-year  demonstration,  an  average  of  14.9  sets  of  criteria  for 
medical  audits  were  developed  in  each  of  the  16  study  hospitals,  although 
this  number  may  include  some  developed  earlier  and  revised  during  1974. 
Patterns  of  care  were  evaluated  for  an  average  of  64  percent  of  these  cri- 
teria sets.     Corrective  actions  were  recommended  for  about  44  percent  of  the 
evaluated  topics.     Of  the  13  hospitals  which  made  recommendations  for  cor- 
rective actions,  54  percent  recommended  education  programs;  69  percent  rec- 
commended  changes  in  organization  policy  or  procedures;  and  85  percent  rec- 
commended  direct  intervention  with  individuals  or  groups.     An  average  of  17 
percent  of  the  topics  recommended  for  corrective  action  were  re-assessed; 
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however,  re-assessments  were  made  in  only  six  hospitals.     Most  of  those 
hospitals  reported  "moderate  improvements"   in  practice,  but  details  were 
not  provided.     An  example  of  improvement  cited  was  a  decreased   incidence  of 
normal  appendices  removed  surgically.    [13]     Some  cost  information  was 
gathered,  but  the  cost  of  conducting  medical  audits  was  not  provided. 

The  attrition  rate  between  initiation  of  the  study  and  completion  of  the 
full  medical  audit  cycle  is  striking  in  both  the  Mandate  Project  and  QAP 
demonstration.     Participating  hospitals  could  receive  technical  assistance 
from  project   staff  in  both  studies,   so  an  inability  to  comprehend  audit 
requirements  should  not  have  been  influential.     The  QAP  demonstration  may  not 
have  permitted  sufficient  time  to  complete  the  full  cycle   for  all  audits, 
but  the  Mandate  Project  covered  a  longer  time  span  and  should  not  have 
suffered  from  this  problem. 

The  difficulties  associated  with  QAP  medical  audits,  as  reported  by  partici- 
pating demonstration  hospitals,   are  similar  to  complaints  voiced  during  the 
site  visits  for  this  study.     Examples  include  difficulty  in  defining  mean- 
ingful topics;  difficulty  in  establishing  concise  and  objective  criteria; 
compromises  in  criteria  development  resulting  in  minimal  rather  than  optimal 
standards;  difficulty  in  achieving  consensus  across  medical  specialties; 
imprecise  screening  guidelines,  producing  too  many  records  for  review;  inade- 
quate documentation  in  medical  records;  excessive  time  devoted  to  abstracting 
process  criteria — particularly  if  outcomes  are  acceptable;  irrelevance  of  out- 
come screens,   since  negative  outcomes  are  usually  explained  by  pre-existing 
conditions;  failure  to  attend  audit  meetings  and  inattention  to  committee 
reports;  difficulty  in  transmitting  corrective  action  needs  among  different 
departments;  and  failure  to  attend  education  programs,  particularly  by  those 
most   in  need  of  improvement. 

The  major  source  of  medical  audits  at  present  are  the  2,500  hospitals  using 
the  JCAH-sponsored  Performance  Evaluation  Procedure   for  Auditing  and  Improv- 
ving  Patient  Care  (PEP),  summarized  in  Appendix  D.     To  facilitate  the  imple- 
mentation of  medical  audits,   the  JCAH  publishes  the  Quality  Review  Bulletin. 
Each  issue  addresses  a  specific  audit  topic,  showing  an  actual  hospital  au- 
dit of  the  topic,  an  analysis  of  the  audit's  strengths  and  weaknesses,  and 
a  minimum  of  two  sets  of  alternate  criteria  used  by  other  hospitals  to  au- 
dit the  same  topic.     Although  the  Bulletin  may  serve  a  useful  educational 
purpose,  there  is  no  equivalent  mechanism  to  systematically  accumulate  suf- 
ficient information  to  permit  assessment  of  the  effectiveness  of  medical 
audits . 

The  JCAH  intends  to  assess  the  "effectiveness  and  efficiency  of  retrospective 
review  of  the  quality  of  care."   [14]     As  currently  planned,   the  report  will 
be  based  on  information  gathered  during  regular  JCAH  hospital  surveys  and 
will  concentrate  on  actions  proposed  to  correct  identified  deficiencies  in 
patient  care  and  subsequent  re-assessment.     It  should  be  available  early  in 
1977.  [15] 


Some  information  about  the  California  Medical  Association's  Patient  Care  Audit 
(PCA)  is  available.     In  April  1975,  the  208  hospitals  that  had  participated 
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in  PCA  workshops  since  their  inception  in  1971  were  surveyed.     The  results 
indicate  that  95  percent  of  the  hospitals  had  implemented  some  PCA  activities; 
73  percent  had  completed  audits  of  one  or  more  topics ;  and  33  percent  had 
completed  a  re-audit  of  one  or  more  topics.     Implementation  was  more  likely 
if  the  PCA  process  was  in  effect  in  the  hospital  before  the  workshop,  the 
hospital  staff  was  committed  to  PCA  and  regarded  it  as  viable,  and  it  con- 
tributed to  improved  patient  care.  [16] 

The  "Audit  Action  Letter,"  published  by  the  Patient  Care  Institute  of  Darien, 
Connecticut,  emphasized  innovative  approaches  to  medical  audits.  Recent 
issues  raised  questions  about  traditional  retrospective  audits  similar  to 
those  experssed  by  administrators  of  programs  examined   in  this  study. 

Despite  the  potential   future  accumulation  of  evaluative  data  about  MCEs , 
today  there  are  no  reliable  data  on  the  numbers,   topics,   and  associated  costs 
of  currently  performed  MCEs;  the   identified  deficiencies  in  patient  care; 
the  remedial  actions  proposed  and  taken;  and  the  extent  and  duration  of  im- 
provements in  patient  care.     MCEs  may  have  improved  quality,  but  reliable, 
be f ore-and-af ter  assessments  are  not  available.     Any  endorsement  of  the  con- 
tinued performance  of  MCEs  must  be  based  on  the  recognition  that  in  isolated 
instances  performance  has  improved;  the  assumption  that  the  questioning  and 
information  exchange  during  conduct  of  audits  may  increase  attention  to 
quality  issues,   thereby  informally  leading  to  improvements;  and  the  hope 
that  future  evaluations  will  provide  more  conclusive  evidence. 

EFFECTS  OF  REVIEW  ON  RESOURCE  UTILIZATION  AND  COST 

More  information  was  available  from  the  programs  visited  about  the  effects 
of  concurrent  hospital  review  and  ambulatory  review  on  utilization  patterns 
and  the  costs  of  review.     However,  the  information  is  still  inadequate. 
Most  programs  have  not  specifically  designed  information  systems  to  permit 
an  evaluation  of  effectiveness.     Reliability  and  specificity  of  data  on 
hospital  admissions  may  have  improved  as  program  staff  gained  an  appreciation 
of  the  need  for  precision  and  an  increased  capability  to  achieve  it,  but  this 
limits  the  utility  of  before  and  after  comparisons.     Political  and  environ- 
mental  influences  may  have  affected  review  in  a  manner  which  would  not  be 
routinely  recorded.     Effectiveness  in  satisfying  unmet  health  needs  is  not 
assessed . 

Cost  data  are  particularly  deficient.     Resource  constraints  precluded  the 
accumulation  of  cost   information  specifically  for  this  study.     Therefore,  the 
data  represent  existing  information  that  could  be  obtained  with  relatively 
little  effort.     Some  data  represent  projections  by  the  program,  but  the 
assumptions  and  methods  by  which  they  were  derived  are  not  clearly  specified. 
Even  "actual"  data  must  be  regarded  as  estimates,   since  the  data  sources  are 
of  recent  origin,  and  uncorrected  errors  may  persist.     Estimates  may  not  in- 
clude depreciation  and  other  indirect  costs;   fixed  investments  may  have  been 
paid  for  by  other  funding  sources.     Although  the  emphasis  is  on  operating 
costs,   some  development  costs  may  be  included,   since  many  systems  are  still 
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evolving.     Variable  costs  and  the  cost  of  alternate  care  are  not  considered. 

In  all  cases,  costs  must  be  regarded  as  under-estimates ,  and  benefits,  as 

over-estimates.     Despite  these  limitations,   the  data  are  probably  the  best 
currently  available. 

HOSPITAL  REVIEW  PROGRAMS 

Information  on  the  costs  of  hospital-based  concurrent  review  programs  from 
three  hospitals  and  three  PSROs  was  analyzed  and  related  to  measures  of 
effectiveness.     Cost  information  from  the  remaining  programs  was  not  analyzed 
because  sufficient  data  were  not  available.     In  some  cases,   anticipated  infor- 
mation was  not  received.     Cost  estimates  are  based  on  the  incremental  re- 
sources required  to  operate  concurrent  review  programs,   regardless  of  whether 
services  are  reimbursable  or  gratis.     The  same  kinds  of  resources  were  ex- 
amined for  all  programs  to  the  extent  that  data  were  available.     In  addition 
to  direct  costs,   incremental  indirect  direct  costs  were  included.  Variation 
among  programs  in  the  techniques  or  procedures  used  to  perform  review  re- 
stricts the  range  of  possible  comparison.     Nevertheless,   the  apparent  pur- 
pose or  intended  outcomes  of  the  programs  reviewed  are  the  same. 

The  estimated  cost  of  concurrent  review  per  hospital  admission  varies:  for 
internal  hospital  systems  during  1976,   the  average  incremental  cost   is  about 
$2.50  at  Bethesda-Lutheran,   $3.50  at  Overlook,   and  $8.00  at  Mt .   Sinai  in 
New  York  City.     PSRO  estimates  for  the  last  half  of  1975  are  $9.76  in  Utah 
and  $9.90  in  Colorado.    [17]     Multnomah  staff  did  not  provide  a  per  admission 
cost  for  concurrent  review,  but  the  cost  of  all  review  components,  including 
gratis  physician  services,   for  the  first  quarter  in  1976  was  reported  as 
$18.69  per  admission.    [18]     Elsewhere  the  concurrent  review  portion  of  total 
costs  is  slightly  over  50  percent. 

A  partial  explanation  for  the  lower  cost  of  internal  hospital  programs  may 
be  that  there  are  more  patients  per  nurse  coordinator,  which  implies  that 
review  is  concentrated  (perhaps  unconsciously)  on  "problem"  cases;  economies 
associated  with  reviewing  all  patients  rather  than  just  those  covered  by 
federal  reimbursement  programs;  and  lower  administrative  or  indirect  costs. 
There  is  some  information  to  suggest  that  the  costs  reported  for  internal 
review  systems  may  increase  as  they  are  integrated  into  a  local  PSRO. 
Additional  work  is  needed  to  adequately  explain  the  reasons  for  variation 
in  costs  and  to  determine  if  equally  effective  but  cheaper  review  mechanisms 
can  be  developed. 

Against  the  background  of  cost  information,  examples  of  changes  in  hospital 
utilization  patterns  thought  to  result  from  concurrent  review  programs  were 
cons  idered . 

Denied  admissions  to  Bethesda-Lutheran  Hospital  resulted  in  an  estimated 
annualized  gross  savings  of  about  $71,000.     The  hospital's  administrator 
estimates  that  variable  expenses  are  about  40  percent  of  the  total,  which 
would  yielu  an  actual  savings  of  about  $28,000 — somewhat  more  than  the 
administrative  cost  of  the  entire  concurrent  review  program.  Information 
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on  the  cost  of  alternate  care  required  by  patients  whose  admissions  were 
denied   is  not  available.     In  any  case,  the  net  savings  are  substantially 
less  than  $28,000.     The  hospital's  average  length  of  stay  declined  between 
1968  and  1972,  but  has  remained  about  the  same  since,  with  a  slight  in- 
crease in  1974.     Although  admissions  and  some  (0.4  percent)  extended  stay 
requests  were  denied  in  1975,  the  stable  average  length  of  stay  admissions 
implies  that  the  impact  was  offset  by  the  elimination  of  the  "easier," 
shorter  stay  admissions  and  use  of  the  "Surgaday"  program,  which  further 
reduced  the  number  of  short-stay  cases. 

Overlook  Hospital  has  reported  a  greater  decline  in  average  length  of  stay 
for  Medicare  patients  between  1970  and  1975  than  that  reported  by  other 
hospitals  in  the   immediate  area  of  Union  County,  New  Jersey.     The  decline 
occurred  primarily  between  1970  (14.9  days)  and  the  second  half  of  1971 
(11.9  days).     In  1975  Overlook's  average  length  of  stay  for  Medicare  patients 
was  11.6  days. 

The  Social  Security  Administration  (SSA)  has  conducted  a  multivariate  analysis 
of  length  of  stay  data  for  community  hospitals   for  the  years  1970,  1971, 
and  1974  (not  for  1972  or  1973)  which  considers  several  factors  such  as 
hospital  characteristics,  typical  mix  of  diagnoses  and  procedures,  and  charac- 
teristics of  Medicare  patients.     The  analysis  for  Overlook  Hospital  showed 
that  the  length  of  stay  in  1970  and  1971  was  not  significantly  different  from 
that  expected  for  the  comparable  class  of  hospitals  in  that  area.   [19]  There- 
fore, the  decline  does  not  appear  to  be  attributable  to  Overlook's  concurrent 
review  program.     Since  the  program  covers  all  patients,   it  is  of  interest 
that  there  was  negligible  change  in  the  length  of  stay  for  non-Medicare 
patients  between  1970  (7.4  days)  and  1971  (7.5  days). 

SSA  reported  that  for  1974  the  average  length  of  stay  for  Medicare  patients 
at  Overlook  was  still  within  the  expected  range,  since  stays  in  other  hospi- 
tals in  that  area  have  also  declined.   [20]     These  data  suggest  that  there 
are  countywide  influences,  such  as  shortage  of  available  beds,  that  may  be 
affecting  length  of  stay.     Indeed,  even  with  reduced  lengths  of  stay, 
Overlook's  occupancy  rate   for  medical  and  surgical  beds  in  recent  years  has 
remained  above  90  percent. 

If  Overlook's  experience  is  considered  independently  of  other  hospitals  in 
the  area,  the  year  1970  becomes  an  appealing  base  for  measuring  effective- 
ness:    gross  savings  of  $2,000,000  would  result  through  1975  from  the  reduc- 
tion of  3.3  days  per  stay  for  about  4,600  federal  pay  patients  at  an  esti- 
mated per  diem  of  $125.     If  variable  expenses  were  20  percent,  savings  would 
still  equal  $400,000  (not  considering  the  cost  of  alternate  care)  or  five 
times  the  estimated  annual  cost  of  concurrent  review  for  all  Overlook 
patients.     However,  if  the  second  half  of  1971  were  used  as  the  base  year, 
such  savings  could  not  be  claimed.     The  program's  medical  coordinator  makes 
no  such  claims. 

Mt .   Sinai  Hospital   in  New  York  City  introduced  a  more  intensive  quality 
assurance  program  in  1975  and  raised  its  associated  annual  budget  from  about 
$50,000  to  $300,000.     A  seven  percent  reduction  in  average  length  of  stay 
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(about  one  day  per  admission)  during  the  12-month  period  following  initiation 
of  the  intensified  program  was  reported.     If  variable  expenses  are  estimated 
at  20  percent,  this  would  yield  savings  of  $1,500,000,   so  review  is  cost 
effective  for  Mt .   Sinai.     The  program's  effectiveness  appears  to  result  from 
more  stringent  requirements  for  patient  treatment  plans  and  more  aggressive 
discharge  planning. 

The  rate  of  change  appears  to  be  narrowing  in  1976,  and  length  of  stay  may 
be  increasing  slightly.     Mt .   Sinai  staff  is  examining  these  statistics  to 
see  whether  they  may  be  influenced  by  changes  in  patient  mix.     Since  the 
inception  of  the  review  program,   patients  admitted  to  Mt .   Sinai  tend  to  be 
more  seriously  ill  and  have  longer  expected  stays. 

Despite  the  change  in  length  of  stay,  the  occupancy  rate  at  Mt .   Sinai  has 
not  declined,  even  though  it  is  located  in  an  area  which  is  overbedded. 
Since  it  is  a  relatively  high-cost   facility,  a  shift  of  patients   from  a 
low-cost  facility  to  Mt .   Sinai  could  mean  a  smaller  amount  of  savings  for 
the  community  than  for  Mt .   Sinai  alone,  assuming  the  same  patient  outcome. 
For  the  community  as  a  whole,  therefore,  the  reduction  in  average  length  of 
stay  at  Mt .   Sinai  may  have  been  associated  with  a  higher  community  expendi- 
ture for  acute  hospital  care,  unless  the  lower  occupancy  elsewhere  resulted 
in  sufficiently  lower  total  costs. 

Adequate  information  for  evaluating  the   impact  of  the  Multnomah  Foundation 
for  Medical  Care  on  hospital  utilization  is  not  available.   [21]  Conditional 
PSRO  activities  were  instituted  recently,  however,  so  the  absence  of  informa- 
tion should  not  necessarily  be  construed  to  imply  a  lack  of  impact.  Given 
the  cost  of  the  Multnomah  review,  a  reduction  in  average  length  of  stay  of 
0.6  days,  or  a  six  percent  reduction  in  a  9.4  average  length  of  stay  per 
admission  (assuming  per  diem  of  $150  and  variable  expenses  of  20  percent) 
would  be  required  to  break  even,  assuming  no  cost   for  alternate  care. 

Using  trend  analysis,  staff  at  the  Colorado  Foundation  for  Medical  Care  has 
estimated  a  total  cost  avoidance  during  1974  of  $9.1  million,  using  full  per 
diem  rates  to  evaluate  each  bed  day  saved.     This  gross  figure  compares  with 
an  approximate  review  cost  of  $1.8  million.     The  Medicaid  trend  is  based  on 
data  from  two  years;  the  Medicare  trend,  on  three  years.     No  rationale  is 
provided  for  the  trends  observed. 

The  reasons  for  the  claimed  savings  are  unclear.     Average  length  of  stay  for 
Colorado  does  not  appear  to  have  declined  significantly.     Admissions  could 
have  been  deterred,  but  available  data  are  insufficient  to  establish  the 
extent  to  which  this  may  have  occurred.     Except   for  increases  in  1972  and 
1973,  Medicare  admissions  have  been  declining  for  some  years.  Data  are 
not  available  for  1975  or  1976.     Additional   information  on  the  source  of 
the  reduction  in  bed  days  of  about  nine  percent  in  one  year  would  help 
to  justify  the  estimates  and  evaluate  the  savings. 

The  CFMC  assumption  that  the  total  per  diem  will  be  saved  for  each  bed  day 
avoided  is  not  valid.     At  most,  only  variable  costs  are  saved.     Fiim  estimates 
of  fixed  and  variable  costs  for  Colorado  do  not  exist.     If  20  percent  of  costs 
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were  variable,   the  value  of  bed  days  saved  (as  estimated  by  CFMC)  would  about 
equal  the  amount  of  CFMC  administrative  costs.     To  the  extent  that  savings  are 
related  to  claimed  reductions  in  admissions,   the  cost  of  alternate  care  becomes 
an  important  factor  in  an  estimate  of  savings.     After  adjustment  for  alternate 
care,   the  cost  comparisons  would  not  be  favorable.     Over  the  longer  term,  if 
40  percent  of  costs  were  variable,   the  cost  of  alternate  care  might  still 
eliminate  much  of  the  savings.     Without  additional   information,   there  is 
insufficient  basis  for  concluding  that  the  CFMC  utilization  review  is  cost 
effective . 

The  Utah  Professional  Review  Organization  stresses  the  quality  improvement 
orientation  of  its  hospital  review  activities.     It  has  not  provided  data  to 
evaluate  changes  in  utilization  or  costs.     Data  are  available  from  the  Social 
Security  Administration  that  describe  the  use  of  services  by  Utah  Medicare 
beneficiaries,  however.     Using  1972  as  the  base  year,   if  the  total  decline  in 
average  length  of  stay  for  Utah  Medicare  beneficiaries  were  exclusively  due  to 
the  UPRO  program,   savings  might  be  equal  to  cost,  without  adjustment  for  the 
cost  of  alternate  care.    [22]     If  a  more  recent  year  were  used  as  a  base,  esti- 
mated savings  would  be  less.     Moreover,  declines   in  average  length  of  stay  were 
reported  for  Utah  before  the  1972  advent  of  concurrent  review  in  a  few  hospitals 
and  for  hospitals  elsewhere  in  the  country  without  a  comparable  concurrent 
review  program.     The  Utah  State  Department  of  Health  believes  that  its  prior 
authorization  program  has  helped  to  control  utilization  by  Medicaid  patients, 
but  this  opinion  is  not  shared  by  UPRO  staff. 

Bonner  has  evaluated  the  UPRO  concurrent  review  program  with  data  provided  by 
the  Educators  Mutual  Insurance  Association  (EMIA).   [23]     Utilization  and  ex- 
penditure patterns  for  two  groups  of  EMIA  subscribers  were  compared  before  and 
after  introduction  of  the  review  program.     About  86  percent  of  admissions  for 
subscribers  in  the  Ogden  area  were  to  hospitals  with  the  review  program.  About 
94  percent  of  admissions  for  subscribers  in  the  Provo  area  were  to  hospitals 
without  the  concurrent  review  program. 

The  analysis  indicates  no  statistically  significant  evidence  of  positive  impact 
from  concurrent  review  on  utilization  as  measured  by  average   length  of  stay, 
admission  rates,  or  days  of  care  per  eligible.     Provo  area  subscribers  had  sig- 
nificantly higher  admission  rates  in  both  time  periods,  but  little  change  be- 
tween periods,  whereas  Ogden  area  subscribers  had  rising  admission  rates.  Ex- 
penditures in  the  Ogden  area  were  significantly  higher  than  in  the  Provo  area 
for  both  ancillary  and  total  charges  after  introduction  of  concurrent  review. 
The  possible  influence  of  differences  in  population  characteristics  on  the  im- 
pact of  review  is  not  examined  in  detail,   nor  are  individual  hospital  differ- 
ences.    Occupancy  rates  in  two  of  the  non-review  hospitals  increased  consider- 
ably during  the  study  period,   and  the  occupancy  rate  in  one  review  hospital 
declined  greatly.     Average  length  of  stay  in  Utah  is  already  low  and  further 
reductions  may  be  difficult  to  achieve.     Nevertheless,  cost  effectiveness  of 
the  Utah  concurrent  review  program  has  not  been  demonstrated. 

The  National  Capital  Medical  Foundation  is  still  in  the  process  of  develop- 
ment and  has  not  accumulated  sufficient  experience  to  show  utilization  or 
cost  trends.     Nevertheless,  of  the  32,813  hospital  discharges  reviewed 
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between  the  implementation  of  the  program  in  the   first  hospital  on  September 
29,   1975,  and  August  31,   1976,   355  admissions  and  continued  stays  were  denied. 
Of  these  denials,  245  are  attributable   to  a  single  hospital,  which  has  parti- 
cipated in  the  program  since  December  15,   1975.     Foundation  officials  also 
report  that  on  any  given  day,   500  administrative  stays  can  be  expected  in 
D.   C.  hospitals.     These  are  stays  in  which  the  patient  does  not  require  acute 
care,  but  does  require   intermediate  or  skilled  nursing  care.     Adequate  alter- 
native care  is  not  available  in  the  District  of  Columbia,  so  the  patient 
remains  hospitalized.     Administrative  stays  comprise  approximately  one-third  of 
the  Title  19,   18,  and  5  hospital  census  and  cost  about  $36.5  million  per  year. 
If  adequate  long-term  care  is  assumed  to  cost  about  one-fourth  that  amount, 
appropriate  placement  of  those  patients  would  yield  about  $27  million  in  gross 
annual  savings.     For  the  short-term,  however,  hospital  administrators  in  the 
District  of  Columbia  may  lack  incentives  to  pursue  this  option,  since  it  would 
lower  hospital  revenues.     The  area  is  overbedded,  and  the  average  occupancy 
rate  is  about  75-78  percent. 

In  general,  available  information  does  not  demonstrate  convincingly  the  cost 
effectiveness  of  the  concurrent  review  programs  visited.  Related  literature 
is  similarly  pessimistic. 

A  1969  study  by  the  Social  Security  Administration  examined  the   impact  of 
Medicare  requirements  for  certification  on  the  14th  and  21st  days  of  hospital 
stay,  using  1966-67  data,  and  concluded  that  more  patients  were  discharged  on 
those  days.   [24]     However,   Slee  and  Cunningham  reported  in  early  1971  that  a 
similar  discharge  pattern  existed  for  persons  of  age  65  and  over  before  imple- 
mentation of  Medicare.   [25]     The  effect  of  certification  was  reconsidered  in 
1975,  based  on  certification  at  the  12th  and  18th  days.     SSA  found  "no  signif- 
icant difference  in  the  patterns.   .    .  no  evidence  of  minor  or  secondary  peaking 
on  the  12th  or  18th  day  ...   In  sum,  even  though  mean  length  of  hospital  stay 
decreased  between  1968  and  1971  under  Medicare,  there  is  no  demonstrable  evi- 
dence that  the  physician  certification  and  re-certification  regulations  influence 
the  distribution  of  short-stay  hospital  discharges  in  that  period."  [26] 

In  contrast  with  SSA's  requirement  that  all  diagnoses  be  re-certified  at  the 
same  time   interval,   the  Pre-Discharge  Utilization  Review  program  (PDUR)  spon- 
sored by  Blue  Cross  of  Western  Pennsylvania  specifies  re-certification  dates 
on  the  basis  of  diagnosis.     (Most  programs  visited  and  the  PSRO  requirements 
also  follow  this  pattern.)     The   impact  of  the  PDUR  program  on  average  length 
of  stay  for  Medicaid  patients  with  14  selected  diagnoses  has  recently  been 
evaluated.     Both  an  initial  study  and  a  larger  replication  included  experi- 
mental and  control  hospitals  matched  for  geographical  location,   size,  teaching 
status,  and  service  mix.     They  produced  similar  findings.     In  the  replication, 
no  differences  resulted  in  average  length  of  stay  which  could  be  attributed 
to  the  review  program  with  the  exception  of  maternity  cases.     The  statistical 
variance  in  average  length  of  stay  also  was  not  reduced.     When  the  data  were 
combined  for  all  diagnoses  and  crude  rates  were  calculated,  average  length  of 
stay  did  decline  after  the  introduction  of  PDUR.     However,  the  reduction  was 
attributed  to  the  decline  in  stays   for  deliveries,  rather  than  for  all 
diagnoses.  [27] 
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These  findings  raise  the  possibility  that  more  refined  analyses  might  detect 
an  impact  from  the  review  programs  visited  if  data  were  adjusted  for  patient 
and  diagnostic  mix.     Fetter  and  Riedel  are  currently  analyzing  the  Colorado 
data  with  the  Autogroup  technique,  which  categorizes  patients  into  homo- 
geneous groups  on  the  basis  of  characteristics  affecting  average  length 
of  stay,  such  as  age,  sex,  diagnoses,  and  complications.   [28]     The  results 
may  shed  light  on  the  potential  impact  of  the  Colorado  hospital  review  pro- 
gram and  suggest  refinements  in  the  review  process  for  other  programs  as 
well . 

Similarly,   it  is  possible  that  a  more  targeted  approach  to  review,  concen- 
trating on  diagnoses,  patients,  or  providers  associated  with  questionable 
patterns  of  care,  would  increase  the  effectiveness  of  quality  assurance  sys- 
tems and  perhaps  be  less  expensive.     Demonstrations  of  such  approaches,  using 
samples  of  cases  rather  than  a  total  population,  are  underway  in  New  Mexico 
and  planned  in  Oklahoma.   [29]     These  evaluations  may  lead  to  greater  efficiency 
and  effectiveness  of  review. 


AMBULATORY  REVIEW  PROGRAMS 

Cost  data  for  ambulatory  care  review  programs  are  even  less  adequate  than 
those  for  hospital  review.     Although  comparisons  among  programs  are  of  ques- 
tionable value,  the  data  provide  examples  of  the  range  of  costs  which  might 
occur  if  these  programs  were  replicated  nationally.     Furthermore,  they  provide 
a  framework  against  which  program  effectiveness  may  be  assessed. 

The  cost  of  manual  review  per  physician  claim  ranges  from  about  $1.50  for  the 
San  Joaquin  Foundation  to  about  $2.50  for  the  Colorado  Foundation's  commercial 
claims  review  and  $4  for  the  Colorado  Foundation's  discontinued  Medicaid  claims 
review.     The  latter  included  only  the  more  complicated  cases  referred  by  the 
intermediary's  screen  to  the  foundation  for  more  intensive  review  and  did  not 
involve  relative  value  scale  (RVS)  adjustment. 

The  cost  of  computer  review  is  usually  lower  than  for  manual  review  if  the 
volume  of  claims  is  sufficient  to  justify  computer  use.     Computerized  Medi- 
caid claims  review  in  New  Mexico  costs  about  $1  per  physician  claim.  [30] 
The  San  Joaquin  Patterns  of  Treatment  program  costs  about  $1.50  per  claim, 
and  the  Utah  PACE  review  costs  about  $1.75  per  claim.   [31]     PACE  review  does 
not  include  RVS  adjustment;  San  Joaquin  does;  in  New  Mexico  the  amount  billed 
is  compared  with  the  50th  percentile  of  billings  for  a  particular  procedure 
by  the  individual  physician  and  the  entire  physician  community  and  paid  at 
the  lower  rate. 

Data  processing  expenses  are  not  the  major  portion  of  cost.     Economies  are 
probably  determined  by  many  factors  including  the  volume  of  claims,  the 
quality  of  claims  data,  efficiencies  of  support  personnel  and  the  sorting 
routines,  exception  rates  (the  number  of  claims  identified  for  more  inten- 
sive, usually  peer,  review  as  a  percentage  of  the  total  number  of  claims), 
and  the  frequency  with  which  exceptions  are  reviewed  (since  special  com- 
parative tabulations  must  be  compiled). 
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Exception  rates  range  from  a  low  of  five  percent  for  both  Colorado  ambulatory 
reviews  to  a  high  of  25  to  31  percent  for  the  San  Joaquin  Patterns  of  Treat- 
ment.    Variation  in  exception  rates  is  difficult  to  interpret,  however,  with- 
out additional  information  on  the  extent  to  which  this  is  an  artifact  of 
review  criteria  or  is  an  accurate  indicator  of  the  amount  of  inappropriate  care. 

The  Colorado  Foundation  for  Medical  Care  reports  that  its  claims  review  has  an 
impact  on  cost  and  utilization  of  ambulatory  care.     Available  information  on  its 
review  for  private  carriers  indicates  a  favorable  benefit-to-cost  ratio.  For 
the  first  nine  months  of  1976,   the  total  cost  of  private  review  was  $101,950. 
The  total  dollar  value  of  claims  reduced  was  $196,499.     Of  this  amount,  $150,086 
resulted  from  RVS  adjustment,   rather  than  peer  review.     Information  is  not 
available  on  the  impact  of  Medicaid  review,  which  has  been  discontinued. 

The  San  Joaquin  Foundation  also  claimed  a  substantial  change  in  utilization 
of  ambulatory  care.     The  extent  of  the  effect  for  all  insurance  carriers  has 
not  been  fully  documented,  but  one  may  assume  that  the  benefits  exceed  the 
payments  for  review.     Some  data  are  available  from  other  sources. 

Buck's  examination  of  the  influence  of  the  San  Joaquin  Foundation  review  on 
physician  practice  patterns  includes  some  information  on  the  costs  of  review 
and  resultant  savings  in  the  costs  of  care.     Between  February  1968  and  May 
1971,  adjustments  to  Medi-Cal  billings  totaled  $1,401,522.     Cost  of  review 
for  that  period  is  estimated  to  be  $195,605  although  this  figure  is  known 
to  under-estimate  actual  costs.     Nevertheless,   the  savings  were  more  than 
adequate  to  offset  costs.     Of  the  total  adjustments  to  billings,  $1,198,739 
were  made  by  claims  examiners,  rather  than  review  physicians  or  the  review 
committee.    [32]     Buck  comments:     "The  savings  listed  under  claims  examiners 
are  partially  due  to  review  based  on  criteria  developed  by  the  (foundation), 
but  they  often  represent  routine  contract  adjustments  that  would  be  realized 
under  most  claims  processing  approaches.     The   .    .    .   savings  as  a  result  of 
physician  review  would  probably  not  be  achievable  without  some  active  partici- 
pation by  physicians."  [33] 

Holahan  was  unable  to  detect  a  significant  difference  between  the  impact  of 
the  San  Joaquin  review  of  Medi-Cal  claims  and  the  California  Blue  Shield 
review,  which  does  not  include  a  provision  for  formal  peer  review.  [34] 
The  two  review  mechanisms  were  not  used  simultaneously  to  monitor  utiliza- 
tion in  the  same  area,  however,   so  a  controlled  comparison  was  not  made. 

Brook  and  Williams  report  that  after  a  two-year  effort,   review  activities 
of  the  New  Mexico  Foundation  decreased  injections  "by  an  estimated  60  percent 
by  modifying  physician  behavior  through  educational  and  direct  sanction  ac- 
tivities.    Many  adverse  reactions  due  either  to  administration  of  the  injec- 
tions (e.g.,   abscesses,  nerve  damage,   or  hemorrhage  from  a  vessel)  or  to 
specific  side  effects  of  the  particular  drug  (death,  disability)  were  most 
likely  prevented."  [35] 

In  addition  to  denials  for  injections,   $43,000  appear  to  have  been  denied  ex- 
plicitly from  physician  claims  for  other  medical  reasons,   and  $133,000  for 
non-medical  (administrative)  reasons.     An  additional  $19,000  in  medical 
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denials  and  $84,000  in  administrative  denials  are  thought  to  have  resulted 
from  claims  for  physician  services,  but  may  have  come  from  other  providers. 
If  these  savings  are  added  to  the  $107,000  from  denied  payments  for  injec- 
tions and  an  estimated  $160,000  from  injections  deterred,  total  savings  would 
be  $546,000.     This  compares  favorably  with  an  estimated  cost  of  ambulatory 
physician  claims  review  of  about  $340,000,  which  may  include  some  develop- 
mental costs.   [36]     The  authors  note  that  injections  received  primary  atten- 
tion from  the  ambulatory  review  process  during  the  first  two  years.  However, 
they  believe  that  "services  rendered  for  many  .   .   .  ambulatory  conditions  are 
clearly  candidates  for  more  peer  review  (using)  explicit  process  criteria  .   .  . 
selected  on  the  basis  of  their  a  priori  relationship  to  health  status."  [37] 

Limited  information  is  available  to  describe  the  impact  of  the  Utah  PACE  pro- 
gram for  ambulatory  care  reivew.     During  the  first  quarter  of  1975,  27  physi- 
cians with  exceptional  patterns  of  care  received  letters  questioning  their 
treatment  methods.     They  accounted  for  311  exceptions.     Sixty-seven  excep- 
tions were  noted  for  the  same  physicians  during  the  first  quarter  of  1976. 
Information  on  improvements  in  specific  exception  rates  is  available,  which 
expresses  the  change  as  the  percent  of  deviant  practice  corrected.     This  method 
tends  to  overstate  the  extent  of  improvement.     Nine  physicians  who  received 
letters  discussing  inappropriate  injection  of  antibiotics  decreased  their 
exception  rates  by  51  percent  in  1976;   inappropriate  injection  of  ACTH  (four 
physicians)  decreased  25  percent;  inappropriate  injection  of  steriods  (two 
physicians)  decreased  19  percent;  excessive  urinalyses  (two  physicians) 
decreased  90  percent;  and  inappropriate  injection  of  estrogen  (four  physicians) 
and  vitamin  B-12  (three  physicians)  and  excessive  visits  (two  physicians) 
were  eliminated.     Multiple  comprehensive  exam  charges  (one  physician)  in- 
creased by  11  percent.     In  each  year,  the  majority  of  exceptions  were  accounted 
for  by  a  few  physicians.     Between  January  and  June  1,  1976,  eight  additional 
physicians  received  letters,  and  one  physician  was  informed  that  further 
services  which  violate  review  guidelines  will  not  be  paid.  [38] 

The  project  manager  expects  that  PACE  will  "produce  savings  by  supplanting 
State  utilization  and  quality  review  efforts  in  ambulatory  care  and  by 
catalyzing  an  alteration  of  practice  patterns  regarding  the  utilization  of 
injections,  visits,  certain  common  lab  and  x-ray  procedures,  and  drugs. 
The  capability  of  PACE  to  identify  any  provider  whose  practice  consistently 
results  in  unnecessary  expense  to  Medicaid  has  been  demonstrated."   [39]  PACE 
has  also  helped  to  identify  cases  of  abuse  by  patients.     As  of  June  1976, 
"24  cases  involving  potentially  serious  patient  abuse  had  been  reported  to 
the  State."     Of  these,  two  or  three   involved  criminal  activity.     For  many  of 
the  cases,  "the  State   is  attempting  to  limit  the  patient  to  a  single  primary 
care  provider  of  the  patient's  choosing."  [40] 

An  examination  of  such  a  limited  number  of  ambulatory  quality  assurance  pro- 
grams cannot  be  considered  a  conclusive  study.     However,  where  common  data 
elements  are  available,  the  findings  are  surprisingly  consistent.     The  dol- 
lar reductions  in  submitted  claims  are  more  than  adequate   to  pay  the  costs 
of  review.     Thus,  review  has  been  demonstrated  to  be  cost  effective  for  the 
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fiscal  intermediary,  but  not  necessa 
exists  when  the  effectiveness  of  the 
programs  is  examined  separately  from 
view  component. 


ily  for  society.  Greater  uncertainty 
medical  peer  review  component  of  such 
the  administrative  or  non-medical  re- 


SUMMARY 

In  this  chapter  the  effectiveness  of  health  care  quality  assurance  programs 
was  considered.     The  discussion  of  the  difficulties  of  measuring  impact  and, 
in  particular,  the  limitations  of  the  data  gathered  during  this  study  should 
not  be  overlooked.     When  cost  data  are  available,  they  are  usually  under- 
estimates.    Claimed  benefits  are  usually  over-estimates.     The  information  is 
not  adequate  to  provide  a  definitive  assessment  of  the  effectiveness  of 
quality  assurance  programs.     However,   it  can  illuminate  current  trends  and 
raise  questions  for  further  research  and  evaluation. 

The  impact  of  review  on  the  quality  of  medical  care  and  health  status  is  by 
far  the  most  difficult  to  assess.     Real  improvements  in  quality  were  de- 
tected— for  example,  the  identification  of  previously  unknown  cases  of  hyper- 
tension.    But  relating  such  an  improvement  to  the  total  magnitude  of  quality 
assurance  activities  and  determining  whether  the  results  justify  the  resource 
expenditures  is  extremely  complex. 

An  assessment  of  the  medical  care  evaluation  or  medical  audit  portions  of 
hospital  review  programs  is  particularly  difficult  for  this  reason.  Although 
accomplishments  were  noted,  program  officials  expressed  serious  reservations 
about  the  value  of  traditional  medical  audit.     Some  are   involved  in  innovative 
attempts  to  devise  more  meaningful  MCEs .     Despite  the  widespread  performance 
of  medical  audits  to  meet  the  accreditation  and  reimbursement  requirements, 
there  are  no  methods  for  accumulating  information  on  both  successful  and 
unsuccessful  MCEs,  assigning  values  to  the  data,  and  aggregating  them  in 
a  manner  which  would  permit  an  overall  assessment  of  effectiveness.     The  Joint 
Commission  on  the  Accreditation  of  Hospitals  plans  to  issue  a  report  on  the 
efficiency  and  effectiveness  of  medical  audits  early  in  1977.     Similarly,  the 
the  Bureau  of  Quality  Assurance  has  awarded  at  least  one  contract  to  explore 
these  issues.     Thus,  better  information  should  be  forthcoming.     At  this  time, 
however,  the  general  effectiveness  of  medical  care  evaluation  studies  has 
not  been  shown. 

Somewhat  better  information  is  available  to  describe  the  impact  of  the  con- 
current review  portions  of  hospital  review  programs.     Program  costs  vary 
widely.     Of  the  programs  visited,  the  costs  of  internal  hospital  systems  are 
usually  lower  than  those  associated  with  PSROs — apparently  because  the  hospi- 
tals assign  more  patients  to  a  single  nurse  coordinator  and  their  administra- 
tive and  indirect  costs  are  lower.     In  both  hospitals  and  PSROs,   the  claims 
for  savings  are  over-estimated,  because  they  erroneously  assume  that  the 
total  per  diem  cost  will  be  saved  for  each  day  denied  and  are  not  properly 
adjusted  for  variable  costs  and  the  cost  of  alternate  care.     Of  the  three 
hospitals  visited,  only  one  has  a  review  program  which  is  clearly  cost  effec- 
tive, although  all  three  have  experienced  changes  in  admission  rates  and 
lengths  of  stay.     Of  the  PSROs,  only  one  has  claimed  significant  savings, 
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and  data  are  not  available  to  accurately  explain  this  phenomenon.     In  general, 
evidence  is  not  yet  available  to  conclude  that  hospital  concurrent  review 
programs  are  effective. 

The  ambulatory  claims  review  programs  for  which  data  are  available  have  shown 
that  dollar  reductions  in  claims  considerably  exceed  the  cost  of  review. 
Although  these  programs  may  be  cost  effective  for  the  fiscal  intermediary, 
the  burden  of  paying  for  denied  claims  may  fall  later  upon  the  patient,  other 
fiscal  intermediaries,  or  society.     When  there  are  savings,   they  tend  to  come 
from  routine  adjustments  that  would  be  realized  under  most  claims  review  sys- 
tems, which  are  unrelated  to  considerations  of  either  appropriateness  or 
quality  of  care.     The  additional  benefit  which  results  from  the  peer  review, 
qualitative  component  of  the  ambulatory  review  systems  is  not  known. 

If  the  current  cost  of  review  per  discharge  for  Medicare  and  Medicaid  were  pro- 
jected to  all  federal  beneficiaries,  current  federal  expenditures  for  hospital 
quality  assurance  activities  would  be  about  $200,000,000.     If  the  anticipated 
level  for  fiscal  year  1977  were  projected  to  a  national  total,   this  would  be 
$275,000,000.     Although  the  amount  includes  some  developmental  costs, 
the  budget  could  well  become  institutionalized  at  that  level.  Another 
$250,000,000  probably  would  be  added  to  monitor  the  ambulatory  care  received 
by  federal  beneficiaries.     This  is  based  on  the  lower  end  of  the  cost  spectrum 
observed  for  current  patterns  of  review  for  Medicare  and  Medicaid  patients. 
If  the  same  data  are  used  to  estimate  the  cost  of  review  for  the  total  popula- 
tion, expenditures  could  reach  $750,000,000  for  ambulatory  review  and 
$500,000,000  for  hospital  review,  or  a  total  of  $1,250,000,000.     Although  the 
sums  are  significant,   the  returns  from  those  expenditures  are  by  no  means 
certain . 


STEERING  COMMITTEE  RECOMMENDATIONS 

1.  Intensified  efforts  should  be  mounted  immediately  to  evaluate  systemat- 
ically both  federal  and  privately  sponsored  health  care  quality  assurance 
systems,  using  careful  evaluative  designs  and  incorporating  control  areas  when- 
ever possible.     The  choice  of  review  mechanisms  for  evaluation  should  not  be 
left  to  chance,  but  should  be  explicitly  planned  in  advance  to  consider  a 
range  of  alternative  approaches.     A  long-term  prospective  evaluation  strategy 
is  needed,   since  any  major  changes  in  provider  and  patient  behavior  may  be 
expected  to  require  a  long  time  to  emerge  and  become  internalized. 

2.  Related  research  is  needed  to  develop  better  evaluative  measures.  In 
particular,   improved  methods  are  needed  to  identify  and  aggregate  the  effects 
on  health  status  which  result  from  the  provision  of  medical  care.  Summary 
measures  are  needed  to  assess  improvements  which  stem  from  quality  assurance 
programs,  continuing  education,   and  other  activities  designed  to  improve  the 
quality  of  care  and,  ultimately,   to  elevate  the  health  status  of  the.  popula- 

t  ion . 

3.  Criteria  should  be  developed  for  categorizing  successful  and  unsuccessful 
medical  care  evaluation  studies  and  isolating  factors  associated  with  success, 
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so  that  more  effective  MCEs  may  evolve.     This  will  require  the  resolution  of 
an  apparent  dilemma.     The  success  of  MCEs  has  been  said  to  rest  on  medical 
staff  involvement  in  selecting  topics,   establishing  criteria,   and  reviewing 
the  results.     Yet,  without  the  involvement  of  an  external  monitor,  such 
efforts  are  potentially  self-serving.     Experimentation  with  a  mixture  of  in- 
ternally and  externally  initiated  MCEs  may  result  in  an  optimal  balance.  A 
wide  range  of  innovation  and  evaluation  should  be  encouraged. 

4.  More  refined  techniques  are  needed  to  determine  the  impact  of  quality 
assurance  programs  on  utilization  and  cost  of  medical  care.  Adjustments 
should  be  made  for  patient,  provider,   and  facility  mix;  and  community-based 
measures  of  effectiveness,   including  the  costs  of  alternative  care,  should 
be  used  whenever  possible.     Additional  research  is  needed  to  determine  the 
conditions  under  which  hospital  costs  vary  according  to  occupancy  rate 

in  both  short  and  long-term  situations.     The  influence  of  cost-plus  reim- 
bursement policies  deserve  special  attention.     Adjustments  for  variable  costs 
and  the  cost  of  alternative  care  should  be  included  in  estimates  of  cost 
savings  resulting  from  utilization  review. 

5.  The  wide  variation  in  the  costs  of  both  ambulatory  and  hospital  quality 
assurance  programs  should  be  further  explored  to  determine  whether  cheaper, 
but  equally  effective  review  mechanisms  can  be  developed.     With  more  exten- 
sive information  to  describe  profiles  or  patterns  of  care,   it  may  be  possible 
to  identify  specific  diseases,  conditions,  providers,  or  patients  associated 
with  inappropriate  patterns  of  care.     A  targeted  review  could  then  focus  on 
those  cases,  rather  than  on  the  total  spectrum  of  care.     This  should  result  in 
more  frequent  identification  of  inappropriate  cases  and  less  frequent  review 

of  appropriate  cases,  which  should  increase  the  efficiency  of  review.  However, 
cases  which  are  excused  from  routine  review  should  be  monitored  periodically 
to  assure  that  more  frequent  review  is  not  warranted.     Evaluation  of  the 
effects  of  targeted  review  must  be  based  on  carefully  coded  diagnostic  data 
with  appropriate  adjustment  for  factors  influencing  treatment  requirements 
and  lengths  of  stay. 

6.  Until  more  extensive  knowledge  of  the  efficacy  of  medical  treatment 
accumulates,   it  may  be  desirable  to  devise  separate  methods  for  cost  and  quality 
control  in  the  few  areas  where  desired  practice  patterns  are  known.  For 

the  remaining  spectrum  of  health  care,   the  lack  of  unanimity  regarding  optimal 
treatment  patterns  may  prevent  the  development  of  national  standards.  One 
would  then  expect  considerable  variation  in  local  criteria  and  practice 
and  little  specific  behavior  change,  which  would  be  accepted  on  the  assumption 
that  greater  involvement  in  and  sensitivity  to  quality  issues  at  the  local 
level  will  raise  the  level  of  care  for  all  conditions  over  time. 


80 


FOOTNOTES 


Chapter  4 


1 

U.S.  Congress,  House  of  Representatives,  Social  Security  Amendments  of 
1972,  Pub.  L.  92-603,   92d  Cong,  2d  sess.,  1972,  H.R.I. 

2 

Alan  R.  Nelson,  "Quality  Assessment  and  Utilization  Review  in  a  Functioning 
Peer  Review  System,"  19  September  1975,  p.  10.  (Mimeographed.) 

3 

David  Bailey  and  Donald  Riedel,  "Recertif ication  and  Length  of  Stay:  The 
Impact  of  New  Jersey's  AID  Program  on  Patterns  of  Hospital  Care,"  Blue  Cross 
Reports  6  (July  1968):  1-10. 

4 

Reported  at  the  Department  of  Health,  Education,  and  Welfare,  National  Center 
for  Health  Services  Research,   Seminar  in  December  1975;  see  also  U.S. 
Department  of  Health,  Education,  and  Welfare,  Public  Health  Service,  Federal 
Employees  Health  Benefits  Program — Utilization  Study,  by  Donald  Riedel  et  al . , 
January  1975.  DHEW  Pub,  No.  (HRA)  75-3125,  Rockville,  Maryland. 

5 

Gerald  Perkoff,  Lawrence  Kahn ,  and  Phillip  Haas,  "The  Effects  of  an  Experi- 
mental Prepaid  Group  Practice  on  Medical  Care  Utilization  and  Cost,"  Medical 
Care  14  (May  1976):  432-49. 

6 

Joseph  Lipscomb,   Ira  Raskin,  and  Joseph  Eichenholz,  "The  Use  of  Marginal 
Cost  Estimates  in  Hospital  Cost  Containment  Policy,"  Milbank  Memorial  Fund 
Quarterly ,  forthcoming. 

7 

U.S.   Department  of  Health,  Education,  and  Welfare,  Office  of  the  Assistant 
Secretary  for  Health,   Office  of  Professional  Standards  Review,  Program 
Evaluation  Plan:     Professional  Standards  Review  Organizations,  by  Martin 
Baum  et  al . ,   (22  September  1975),  p  37. 


81 


8 

The  term  medical  care  evaluation  study  (MCE)  is  used  interchangeably  with 
medical  audit. 

9 

Charles  Buck,   "Peer  Review:     The  Impact  of  a  System  Based  on  Billing 
Claims,"  (Sc.D.  dissertation,  The  Johns  Hopkins  University,  1972),   p.  92. 
See  also  Charles  Buck  and  Kerr  White,   "Peer  Review:     Impact  of  a  System 
Based  on  Billing  Claims,"  New  England  Journal  of  Medicine  291  (24  October 
1974):  877-83. 

10 

Daniel  Fleischer  et  al . ,   "The  Mandate  Project:     Institutionalizing  a  System 
of  Patient  Care  Quality  Assurance,"  n.d.  (Mimeographed.) 

11 

The  bi-cycle  concept  is  an  approach  to  medical  audit  which  includes  a 
program  of  change  to  correct  identified  deficiencies  and  subsequent  re- 
assessment to  determine  the  amount  of  improvement.     This  approach  is  incor- 
porated in  the  PSRO  and  JCAH  MCE  requirements.     It  is  further  described 
in:     Clement  Brown,  Jr.   and  Henry  Uhl,   "Mandatory  Continuing  Education, 
Sense  or  Nonsense?",  Journal  of  the  American  Medical  Association  213 
(7  September  1970):  1660-68. 

12 

Hospital  Research  and  Educational  Trust,  American  Hospital  Association, 
"Final  Report  on  the  Quality  Assurance  Program  Demonstration  Project"  to 
the  W.  K.  Kellogg  Foundation,  Chicago,  1975,  p.  11. 

13 

Ibid.,  pp.  34-42. 
14 

Department  of  Health,  Education,   and  Welfare,  Minutes  of  Meeting  of  the 
National  Professional  Standards  Review  Council,  3-4  May  1976,   comments  by 
John  Porterfield. 

15 

John  Porterfield,  Joint  Commission  of  Accreditation  of  Hospitals,  Chicago, 
telephone  conversation,  23  July  1976. 

16 

California  Committee  on  Regional  Medical  Programs,   "Summary  Report  on 
Evaluation  of  the  Implementation  of  CMA/CHA  Patient  Care  Audit  in  California 
Hospitals,"  April  1975.  (Mimeographed.) 

17 

For  consistency,   cost  figures  for  Utah  and  Colorado  were  taken  from  Bureau 
of  Quality  Assurance  forms  121  and  151,   and  indirect  costs  were  redistributed. 
Staff  of  the  Colorado  Foundation  state  that  this  figure  over-estimates  the 
per  admission  cost  because  the  CFMC-PSRO  information  system  does  not  capture 
data  on  patients  who  are  admitted  in  one  quarter  and  discharged  in  the  next 
quarter.     This  discrepancy  may  exist  elsewhere,   as  well. 


82 


18 

David  A.   Stewart,  Assistant  Executive  Director,  Multnomah  Foundation  for 
Medical  Care,  personal  letter,  26  April  1976.     BQA  figures  comparable  to 
those  for  Utah  and  Colorado  could  not  be  derived  because  of  difficulty 
in  capturing  costs  for  delegated,  non-delegated,  and  potentially  delegated 
hospitals . 

19 

U.S.  Department  of  Health,  Education,  and  Welfare,  Social  Security 

Administration,  Office  of  Research  and  Statistics,  MADOC-4  (July-December 
1970)  and  MADOC-5  (July-December  1971). 

20 

U.S.  Department  of  Health,  Education,  and  Welfare,  Social  Security 
Administration,  Office  of  Research  and  Statistics,  preliminary  data  from 
MEDPAR-1  (fiscal  year  1974)  not  yet  released. 

21 

Some  utilization  data  from  the  Multnomah  Foundation  was  received  on 
September  28,  1976.     However,  it  was  not  possible  to  analyze  the  informa- 
tion adequately  and  explain  the  inconsistencies  in  time  to  include  the 
data  in  this  report. 

22 

Idem,  Current  Utilization  Tabulations  of  Length  of  Stay  Based  on  20%  of 
Medicare  Discharges,  for  six-month  periods  (tabulations  used  for  internal 
SSA  purposes ) . 

23 

Paul  Bonner,  "On-site  Concurrent  Hospital  Utilization  Review:     An  Evaluation 
of  Impact  on  Utilization  Patterns  and  Expenditures,"   (Sc.D.  dissertation, 
Harvard  School  of  Public  Health,  February  1976). 

24 

U.S.  Department  of  Health,  Education,  and  Welfare,  Social  Security 
Administration,  Office  of  Research  and  Statistics,  Use  of  Hospital  Services 
Under  Medicare:     Length  of  Stay  of  Patients  Discharged  from  Short-Stay 
Hospitals,  July  1966-1967,  by  Aaron  Krute,  Health  Insurance  Statistics 
Report  No.  Hl-10  (30  January  1969). 

25 

U.S.  Department  of  Health,  Education,  and  Welfare,   Social  Security 
Administration,   Office  of  Research  and  Statistics,  Utilization  of  Short- 
Stay  Hospitals  Under  Medicare,   1968-1971,  by  George  Chulis,  DHEW  Pubn.  No. 
(SSA)  75-11702  (1975),  citing  Vergil  Slee  and  Raymond  Cunningham,  "Discharge 
Patterns  for  Patients  65  and  Older:     Pre-  and  Post-Medicare,"  PAS  Reporter 
9  (25  January  1971). 


26 

Idem,  p.  8. 


83 


27 

M.  Karlene  Clendenning  et  al . ,  "The  Effect  of  a  Target-Based  Utilization 
Review  Program  on  Length  of  Stay,"  Medical  Care,  forthcoming. 

28 

Donald  Riedel,  "Implementation,  Evaluation,  and  Extension  of  Patient  Care 
Monitoring  Mechanism,"  Grant  #240-75-0051,  DHEW,  PHS,  Bureau  of  Quality 
Assurance,  in  progress. 

29 

"Oklahoma  Utilization  Review  System  Proposal,"  Office  of  Quality  Standards, 
DHEW,  to  begin  shortly.     See  also,  Health  Care  Management  Systems,  Evaluation 
Report;     New  Mexico  PSRO  Revised  Medicaid  Ambulatory  Care  Review  System, 
La  Jolla,  California,   (August  1976) . 

30 

The  estimate  of  $1  per  physician  claim  was  derived  from  information  reported 
in  Robert  Brook  and  Kathleen  Williams,  "Evaluation  of  the  New  Mexico  Peer 
Review  System  1971-1973,"  Medical  Care,  forthcoming. 

31 

Additional  PACE  cost  information  was  received  after  the  termination  of 
this  contract.     It  was  not  possible  to  analyze  this  material  in  time  to 
include  it  in  this  report,  but  an  attempt  will  be  made  to  include  it  in 
the  forthcoming  paper  on  cost  and  cost  effectiveness  of  health  care  quality 
assurance  programs.     To  obtain  the  more  recent  information  the  reader  is 
referred  directly  to  the  Utah  Professional  Review  Organization. 

32 

Buck,  "Peer  Review,"  pp.  89-91. 
33 

Ibid.,  p.  92. 

34 

John  Holahan,  Physician  Supply,  Peer  Review,  and  Use  of  Health  Services  in 
Medicaid  (Washington,  D.C.:  Urban  Institute  (February  1976),  pp.  63-64. 

35 

Brook  and  Williams,  "New  Mexico  Peer  Review,"  p.  102  of  manuscript. 
36 

Ibid.     The  cost   figures  are  taken  directly  or  derived  from  the  manuscript 
as  follows:  $43,000  (derived)  in  explicit  denials  of  physician  claims  for 
medical  reasons  (p.  41);   $133,000  (derived)  for  administrative  denials 
(p.  63);  $19,000  (derived)  and  $84,000  (derived)  in  claims  from  physicians 
or  other  providers  (p.  63);   $107,000  from  injections  denied  (p.  71); 
$160,000  from  injections  deterred  (p.   90);  and  $340,000  (derived)  estimated 
cost  of  review  (p.  103  and  p.  211). 


84 


37 

Ibid.,  p.  222. 
38 

James  Q.  Cannon,  PACE  Project  Manager,  personal  letter,  28  June  1976. 
39 

Idem,  personal  letter,  2  April  1976. 
40 

Idem,  personal  letter,  28  June  1976. 


Chapter  5 

PRIORITY  AREAS  FOR  QUALITY  ASSURANCE 


Several  topics  were  defined  as  priority  areas  because  of  their  crucial 
role  in  determining  the  effectiveness  of  quality  assurance  programs  and  the 
absence  of  reviews  that  integrate  and  analyze  relevant  literature.  The 
conclusions  and  recommendations  of  the  steering  committee  for  five  areas 
are  presented  in  this  chapter. 

The  areas  overlap  somewhat.     In  particular,  outcome-oriented  quality  assur- 
ance programs  are  difficult  to  examine  separately  from  the  health  care  set- 
tings in  which  they  are  found.     For  that  reason  a  separate  paper  was  not  pre- 
pared,  and  the  discussions  of  quality  assurance  for  ambulatory  and  long-term 
care  include  examples  of  outcome  assessment.     The  conclusions  and  recommenda- 
tions for  outcome  assessments  are  below,  preceding  the  summaries  of  ambulatory 
and  long-term  care  quality  assessment.     The  chapter  concludes  with  a  summary 
of  the  implications  of  quality  assessment  for  improving  provider  performance 
and  a  discussion  of  consumer  involvement  in  quality  assurance  programs. 

ASSESSMENT  OF  HEALTH  OUTCOMES 

The  term  "health  outcome"  usually  refers  to  the  end  results  of  medical  care 
measured  by  health  status  and  patient  satisfaction.    [1]     The  assessment  of 
health  outcome  as  an  indicator  of  the  quality  of  care  was  advocated  by 
Florence  Nightengale  in  the  mid-1800s,    [2]  and  more  recently  by  the  U.S. 
Congress,  which  included  in  the  Health  Maintenance  Organization  Act  of  1973 
the  requirement  that  each  participating  HMO  provide  "an  ongoing  quality 
assurance  program  for  its  health  services  which  .    .    .     stresses  health  out- 
comes . "  [  3] 

Outcome  assessment  is  appealing,   since  the  practice  of  medicine  is  based  on 
the  assumption  that  treatment  benefits  the  patient.     Furthermore,   as  concerns 
for  the  cost  of  care  mount,  health  outcomes  may  increasingly  be  used  to 
measure  the  return  on  the  investment  in  medical  care.     Examples  of  the  routine 
assessment  of  health  outcome  as  an  indicator  of  quality  are  somewhat  limited, 
however. 
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Data  on  mortality  and  morbidity  gathered  by  the  U.S.   Census  Bureau  and  the 
National  Center  for  Health  Statistics  show  regional  and  national  trends 
and  are  useful  for  disease  surveillance.     However,   the  time  lag  between 
the  receipt  of  care  and  the  reflection  of  changes  in  morbidity  or  mortality 
statistics  may  be  substantial.     In  addition,   the  data  are  usually  aggregated 
in  a  manner  which  makes  it  difficult  to  identify  specific  patient  or  provider 
groups  associated  with  unusual  trends  who  could  benefit  from  a  case-by-case 
quality  assessment.     Both  factors  limit  the  utility  of  national  mortality 
and  morbidity  data  for  an  ongoing  quality  assurance  system  as  envisioned 
by  this  study. 

Information  on  the  incidence  and  outcome  of  some  specific  diseases  (partic- 
ularly malignancies  and  other  catastrophic  conditions  such  as  end-stage 
renal  disease)  is  captured  on  a  case-by-case  basis  by  disease  registries, 
which  may  also  include  information  on  the  treatment  process.  Registries 
are  usually  designed  for  epidemiologic  research,  however,  rather  than  quality 
assurance  activities,  and  are  probably  too  expensive  for  routine  use  in 
quality  assurance. 

Despite  the  limitations  of  the  more  readily  available  mortality  and  morbidity 
data,   in  some  cases  they  have  been  used  successfully  in  quality  assessments. 
Shapiro  compared  prematurity  and  perinatal  mortality  rates  for  women  enrolled 
in  the  Health  Insurance  Plan  of  New  York  City  (HIP)  and  women  in  the  general 
population,  using  data  on  live  births  and  fetal  and  infant  deaths  from  the  New 
York  City  Department  of  Health.   [4]     Kessner  has  applied  the  same  data  base 
more  recently  in  his  study  of  infant  deaths   in  relation  to  the  need  for  and 
receipt  of  health  services  by  pregnant  women  in  New  York  City.   [5]  Hospital 
discharge  abstracts  processed  by  the  Commission  on  Professional  and  Hospital 
Activities'  Professional  Activity  Study  (PAS)  have  been  used  to  compare 
surgical  mortality  rates  in  1,224  hospitals  included  in  the  "Institutional 
Differences  Study"  conducted  by  the  Stanford  Center  for  Health  Care  Research.  [6] 

One  problem,  however,   is  the  difficulty  of  controlling  for  the  many  factors 
which  influence  outcome,   including  the  patient's  general  health,  demographic 
characteristics,  social  and  economic  factors,  and  structural  aspects  of  the 
health  care  setting.     These  factors  are  particularly  important  if  one  plans 
to  use  unacceptable  outcomes  to  identify  specific  deficiencies  in  the  pro- 
cess of  care.     Shapiro  has  written  of  the  complexities  of  designing  and 
analyzing  end-result  studies.    [7]     The  importance  of  controlling  for  patient 
characteristics  and  surgical  procedures  in  the  Institutional  Differences 
study  is  discussed  by  Moses  and  Mosteller.    [8]     Mushlin  et  al .   studied  out- 
comes for  patients  with  upper  respiratory  infection,   sore  throat,   and  urinary 
tract  infection  and  found  that  one  reason  for  failure  to  achieve  anticipated 
outcome  was  the  presence  of  complicating  factors  related  to  the  patient,  in- 
cluding past  history  and  comorbidity.   [9]     More  recently,  Romm  et  al .  have 
reported  that  the  most  significant  predictor  of  outcome  status  for  patients 
with  congestive  heart  failure  is  "the  patient's  initial  status"  and  that 
"certain  aspects  of  the  process  of  care  have  little  measurable  impact."  [10] 

Shorter  term,  intermediate  or  proximate  outcomes,  each  of  which  results  from 
a  previous  step  in  the  diagnostic  and  treatment  process,  can  be  derived  from 
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Williamson's  definition  of  health  outcome:     "...   any  characteristic  of 
the  patient,   the  health  problem,   the  provider,  or  their  interaction 
in  the  care  process  which  results  from  medical  care  provided  or  required, 
measured  at  one  point  in  time."   [11]     The  JCAH  requirements  for  medical 
audit,   in  particular,   incorporate  an  assessment  of  proximate  outcomes, 
generally  defined  as  complications  in  hospitalized  patients  or  performance 
of  critical  therapeutic  procedures,  which  are  regarded  as  outcomes  of  the 
diagnostic  decision-making  process.    [12]     If  surgical  procedures  are  considered 
outcomes  of  clinical  decisions,  Lembcke's  studies  of  variations  in  appendectomy 
and  hysterectomy  rates  would  fall   into  this  category.    [13]  Similarly, 
Starfield  and  Scheff  assessed  the  proximate  outcome  of  hemoglobin  level 
in  children  in  a  hospital  clinic.  [14] 

Proximate  outcomes  can  be  measured  while  the  patient  is  receiving  treatment 
or  shortly  thereafter  and  do  not  have  the  problems  associated  with  longer 
range  outcomes — the  passage  of  time  which  makes  it  difficult  to  identify 
factors  that  cause  particular  outcomes  and  the  time  and  expense  necessary 
to  locate  patients  and  their  medical  records.     Proximate  outcomes  usually 
do  not  project  far  enough  into  the  future  to  permit  direct  assessment  of 
the  end-results  of  treatment  as  determined  by  health  status  and  ability  to 
function.     Ideally,  however,   final  outcome  is  correlated  with  proximate 
measures  such  as  whether  an  appendix  was  perforated  at  the  time  of  surgery, 
the  APGAR  score  of  an  infant,   lead  level  in  children,   and  blood  pressure. 

Some  research  and  demonstration  projects  have  attempted  to  design  and  evaluate 
quality  assurance  programs  which  incorporate  outcome  assessments.  Williamson's 
earlier  work  has  evolved  into  "health  accounting,"  which  is  designed  to 
achieve  and  document   improved  patient  health  or  reduced  resource  utilization. 
In  implementing  health  accounting,  a  clinic  priority  team  selects  study  topics 
based  on  prevalent  health  problems  for  which  outcomes  could  be  improved.  The 
objective  is  to  maximize  the  achievable  benefit  not  achieved  (ABNA).     A  study 
design  team  estimates  impairment  levels  which  should  be  expected  in  patients 
with  the  study  conditions,  depending  on  the  extent  of  treatment  and  varying 
patient  prognoses,   and  develops  a  plan  for  the  study.     A  health  accountant 
gathers  and  arrays  data  to  compare  estimated  and  actual  outcome.     The  analysis 
includes  an  assessment  of  diagnostic  outcome  (considering  both  false  negative 
and  false  positive  diagnoses)  and  therapeutic  outcome  (measured  on  a  six-point 
functional  assessment  scale).    [15]     Examples  of  health  accounting  have  been 
described  in  the  literature,    [16]  and  a  major  demonstration  is  under  way. 

Ressner's  "tracer"  method  is  another  which  assesses  quality  by  examining 
health  outcomes.    [17]     As  orginally  conceived,   the  tracer  method  viewed  the 
prevalence  of  specified  diseases  or  conditions  (detected  by  physical  exam) 
within  a  defined  community  as  an  outcome  of  health  care.     Initially,   it  was 
assumed  that  the  quality  of  care  for  tracers  would  be  representative  of  the 
overall  quality  of  care  delivered  by  an  individual  provider  or  health  care 
organization. 

The  health  accounting  and  tracer  methods  are  conceptually  appealing,  since 
they  are  based  on  an  epidemiologic  approach.     However,   recent  attempts  to 
apply  them  have  led  to  questions  about  some  basic  assumptions.    [18]  The 
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demand  on  resources  (particularly  for  more  highly  trained  personnel  than 
initially  acticipated) ,   the  inadquacy  of  medical  documentation,   and  limited 
generalizability  of  findings  suggest  that  the  techniques  may  be  appropriate 
for  special  studies.     However,   additional  refinements  are  needed  before  they 
could  be  instituted  as  routine  quality  monitoring  systems — particularly  in 
small,   fee-for-service  office  practice. 

The  work  cited  above  suggests  that  in  the  near  future,   it   is  unlikely  that 
comprehensive  outcome  assessments  will  be  included  in  formal  quality  assur- 
ance programs.     This  may  change  as  a  result  of  some  ongoing  and  recently 
completed  research.     In  one  portion  of  the  Institutional  Differences  Study, 
routinely  gathered  information  on  the  PAS  abstract  formed  the  basis  for 
sophisticated  statistical  estimates  of  the  risk  of  death  for  each  patient; 
the  estimates  were  used  to  compute  indirectly  standardized  mortality  ratios 
for  study  hospitals.     The  authors  believe  that  with  refinements,  their 
techniques  could  be  used  to  develop  a  system  for  "computing  valid  and  reli- 
able measures  of  the  quality  of  surgical  care  in  hospitals   .   .   .  that  would 
depend  primarily  on  PAS-like  data."   [19]     Detmer  is  attempting  to  correlate 
process  and  outcome  measures  for  trauma  patients  in  Wisconsin  in  hopes  of 
developing  proxy  outcome  measures.   [20]     Brook  et  al.  recently  published 
explicit,  short-term  outcome  measures  developed  by  expert  panels  for  eight 
disease  conditions.     For  each  condition  the  report  includes  a  discussion  of 
factors  not  influenced  by  medical  care  that  might  adversely  affect  outcome 
and  proposed  methods  of  controlling  these  factors  in  analyzing  outcome  data. 
The  measures  have  not  been  tested,  however.  [21] 

It  will  be  some  time  before  the  results  of  these  efforts  can  be  easily  applie 
in  a  variety  of  health  care  settings.     Nevertheless,   it  is  possible  to  begin 
immediately  to  assess  the  quality  of  care  on  the  basis  of  outcomes  if  simpler 
but  nevertheless  meaningful,  steps  are  taken. 


STEERING  COMMITTEE  RECOMMENDATIONS 

1.  Existing  techniques  to  monitor  the  progress  of  patients  over  time  should 
be  used  in  quality  assessment  programs.     Most  have  been  developed  primarily 
for  chronic  diseases  and  have  not  been  widely  applied  in  quality  assurance, 
but  their  potential  is  readily  apparent.     Several  instruments  have  been 
developed  to  measure  the  patient's  ability  to  function  in  daily  life.  [22] 
These  may  be  viewed  as  outcome  measures  which  do  not  have  the  limitations  of 
mortality  and  morbidity  statistics.     Some  have  been  evaluated  to  determine 
reliability,  required  resources,  and  associated  costs.     Similarly,  increasing 
emphasis  is  being  placed  on  describing  the  natural  history  of  disease,  speci- 
fic stages  of  the  disease  process,  and  expected  levels  of  functioning  at  each 
point  along  the  spectrum.    [23]     Both  bodies  of  information  should  be  incorpo- 
rated into  ongoing  quality  assurance  programs  as  a  guide  to  plan  for  health 
care  needs  and  to  determine  the  extent  to  which  each  patient's  functional 
ability  is  in  accord  with  expectations. 

2.  In  the  same  way  that  process  measures  of  quality  are  required  of  PSROs, 
limited  outcome  information  after  discharge  should  also  be  gathered  for 
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special  studies.     The  cost  of  these  studies  should  be  carefully  monitored  to 
assure  that  the  benefits  outweigh  the  costs.     Outcome  information  might 
identify  patients  and  providers  for  more  in-depth  assessment.  Variations 
in  outcome  using  different  treatment  methods  may  identify  areas  for  efficacy 
studies.     The  accumulated  data  should  lead  to  a  better  understanding  of  the 
natural  course  of  illness.     Eventually,   sufficient  knowledge  should  be 
accumulated  so  that  if  patients  of  a  particular  provider  have  not  progressed 
as  expected,   the  provider's  treatment  methods  could  be  questioned  or  the 
patient  referred  elsewhere  for  evaluation  and  consultation. 

3.  Assessment  after  discharge  should  be  required  for  all  patients  and  all 
facilities,  not  just  federally  reimbursed  patients. 

4.  Individual  practitioners  should  be  encouraged  to  join  with  their  patients 
to  establish  outcome  objectives  for  patient  care  and  examine  reasons  for 
failure  to  meet  them. 

5.  The  curricula  of  health  professionals  should  include  course  content  de- 
voted explicitly  to  health  care  evaluation  which  can  be  applied  throughout 
their  careers. 

6.  Additional  research  is  needed  to  establish  the  natural  history  of  dis- 
eases and  the  efficacy  of  medical  procedures  and  therapeutic  regimens.  For 
research  findings  to  be  useful  in  assessing  the  quality  of  care,  determina- 
tions of  efficacy  should  be  made  under  average  as  well  as  ideal  treatment 
situations  at  various  points  in  time  and  should  include  a  broad  range  of  out- 
come measures. 

7.  Over  the  long  term,  medical  practice  should  be  re-oriented  to  emphasize 
health  outcomes.     Despite  the  sometimes  questionable  relevance  of  medical 
education  to  practice,  one  of  the  logical  places  to  begin  this  re-orientation 
is  in  medical  school.     Medical  education  is  currently  oriented  toward  the 
technical  content  of  unrelated  episodes  of  illness,   rather  than  toward 
health  throughout  a  lifetime.     Physicians  should  be  educated  to  have  an 
awareness  of  the  natural  course  of  disease,   to  view  patients  at  any  point 

in  time  as  being  in  one  particular  phase  of  this  natural  course,   and  to 
plan  for  care  and  assess  the  results  of  care  along  this  spectrum.  They 
should  realize  that  their  responsibility  does  not  end  when  the  patient 
leaves  the  office  or  hospital,  but  that  continued  monitoring  is  necessary 
and  that  social,   psychological,   and  environmental  factors  are  as  important 
to  the  patient's  health  as  biomedical  factors.     The  reward  structure  of 
medicine  is  similarly  skewed.     Physicians  are  not  usually  promoted  to  depart- 
ment chairmen,  elected  to  offices  in  medical  societies,  or  reimbursed  on 
the  basis  of  the  functional  status  of  their  patients.     Changes  in  the  incentive 
structure  to  appropriately  emphasize  the  health  of  patients  may  eventually 
improve  the  quality  of  care. 

QUALITY  ASSURANCE  FOR  AMBULATORY  CARE 

Ambulatory  care  may  be  defined  as  those  health  services  provided  to  indi- 
viduals not  confined  to  bed.     It  is  fundamentally  different  from  acute 
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hospital  care,   and  these  differences  must  be  considered  in  designing  ambulato 
quality  assurance  programs.     The  provision  of  ambulatory  care  is  complex  and 
diverse,  permitting  a  wide  range  of  discretion  on  the  part  of  the  health  care 
provider.     Many  different  functions  are  performed  under  this  broad  umbrella, 
each  of  which  may  require  a  different  approach  to  quailty  assessment. 

Primary  care  and  secondary  and  tertiary  consultations  are  included  within 
the  ambulatory  spectrum.     Although  the  functions  overlap,  primary  care,  in 
particular,   includes  well-patient  care;  a  broad  range  of  ill-defined  signs, 
symptoms,  and  conditions;  care  for  acute,  sometimes  self-limiting  episodes  of 
illness;  and  continuing  care,   including  that  of  patients  with  common  chronic 
conditions.     No  existing  assessment  technique  can  accommodate  this  range  of 
functions. 

An  adequate  description  of  the  manner  in  which  ambulatory  care  is  delivered 
is  not  available.     The  studies  cited  below  provide  information  on  patterns 
of  care  for  the  providers  evaluated — usually  group  practices  or  a  limited 
number  of  independent  practitioners  within  a  geographic  area.     Many  of  the 
studies  are  outdated.     Among  the  more  recent  are  Payne  and  Lyons'  assessment 
of  ambulatory  care  in  Hawaii,   [24]  the  American  Association  of  Family 
Physicians-University  of  North  Carolina  study  of  general  practitioners  in 
the  Fort  Wayne  area,   [25]  R.  L.  Riedel's  evaluation  of  patterns  of  care  in 
hospital  outpatient  clinics  and  private  offices  in  the  Greater  Hartford  and 
New  Haven  areas,    [26]  and  the  National  Ambulatory  Medical  Care  Survey.  [27] 
Nevertheless,  without  a  more  comprehensive  assessment  of  existing  deficits 
in  care,   it  is  difficult  to  determine  the  amount  of  resources  which  can  justi 
fiably  be  allocated  to  quality  assurance. 

Furthermore,   in  the  absence  of  convincing  evidence  of  efficacy  of  treatment 
or  a  national  reimbursement  scheme,  it  is  difficult  to  know  what  action  is 
warranted  when  varying  treatment  patterns  are  identified.     Patients  may  seek 
out  physicians  whose  orientation  toward  medical  intervention  is  similar  to 
their  own.     Unless  the  patient  or  society  suffers,   it  may  be  difficult  to 
justify  attempts  to  achieve  conformity  with  whatever  may  be  the  usually  ac- 
cepted treatment  of  choice. 

In  addition  to  the  physician  and  patient,  other  people,  organizations,  and 
facilities  influence  the  quality  of  care  and  must  be  included  in  quality 
assurance  efforts.     Of  particular  importance  are  the  small  independent  lab- 
oratories and  radiology  services  for  which  quality  and  quality  control  are 
so  problemmatic .     Although  techniques  exist  that  could  be  used  to  monitor 
the  quality  of  services  provided  by  such  organizations,  they  are  seldom 
and  non-systematically  applied.     Problems  in  the  structure  of  the  care  set- 
ting may  influence  quality.     These  include  supervision  of  aides  performing 
such  tasks  as  taking  blood  pressures,  routine  re-filling  of  prescriptions, 
and  poor  communication  and  recording  skills  which  adversely  affect  both 
working  relationships  among  office  personnel  and  documentation  in  the  medical 
record  and  other  data  sources.     Access  to  and  links  among  varying  levels  of 
care  are  yet  another  area  for  quality  assessment. 
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Important  data  limitations  include  the  inadequacy  of  current  diagnostic  and 
procedural  terminology — particularly  for  classifying  signs  and  symptoms, 
difficulties  in  linkage  of  information  from  multiple  visits  and  episodes  of 
illness,  uncertainty  about  the  range  of  information  to  be  included  and, 
in  particular,  the  role  of  telephone  calls  and  home  visits  and  problems  in 
capturing  information  from  such  encounters,  difficulty  of  tracing  patients 
among  multiple  providers,   the  cost  of  correcting  data,  and  unresolved  issues 
about  data  confidentiality.     Considerable  resources,   primarily  time  and 
trained  personnel,   are  necessary  to  conduct  quality  assessment   in  ambulatory 
settings.     As  data  requirements  become  more  demanding,   the  skills  needed 
to  retrieve  data  generally  increase  to  a  level  that   few  small  offices  can 
support . 

Despite  these  difficulties,   a  substantial  body  of  literature  describes  ac- 
tivities related  to  quality  assurance  in  ambulatory  care.     Existing  review 
techniques  seem  to  be  determined  by  considerations  of  practicality,  resource 
constraints,   and  leverage  points  within  the  delivery  setting,   rather  than 
by  the  unique  aspects  of  ambulatory  care. 

Many  of  the  early  assessments  of  ambulatory  care  were  conducted  in  prepaid 
group  practices  or  neighborhood  health  centers  and  concentrated  on  the  process 
of  care  with  some  attention  to  outcome  and  structural  factors.     These  studies 
include  a  series  by  Makover,  Daily  and  Morehead  within  HIP  clinics;  [28] 
Shapiro's  studies  comparing  the  care  received  by  HIP  enrollees  with  care 
received  by  comparable  populations  in  other  settings;   [29]  a  nationwide 
survey  of  clinical  services  provided  in  multi-specialty  medical  groups; 
[30]  and  more  recent  comparisons  of  the  utilization  and  quality  of  services 
provided  under  alternative  delivery  arrangements.  [31] 

Many  are  one-time,   special  studies,  but  a  few  have  been  incorporated  into 
routine  quality  assurance  systems.     The  Health  Insurance  Plan  of  Greater 
New  York,   for  example,  has  expanded  its  earlier  efforts  into  a  regular  sys- 
tem which  examines  both  process  and  structure.     Charts  for  review  are  random- 
ly selected  from  information  provided  on  encounter  forms  and  laboratory  re- 
ports.    The  internal  HIP  structural  evaluation  reviews  such  factors  as  appoint- 
ment waiting  times  and  procedures  for  handling  consumer  grievances.     It  is 
supplemented  by  an  external  assessment  conducted  by  the  New  York  City 
Department  of  Health.  [32] 

Morehead 1 s  work  at  HIP  has  been  expanded  into  a  more  comprehensive  assess- 
ment of  services  provided  by  community  health  centers  (formerly  known  as 
neighborhood  health  centers  and  funded  originally  by  the  Office  of  Economic 
Opportunity).     Her  methods  have  been  used  for  comparisons  among  community 
health  centers   [33]  and  also  to  compare  the  centers  with  other  health  care 
providers.    [34]     Two  series  of  assessment  were  conducted.     The  second  assess- 
ments indicated  that  73  percent  of  the  centers  showed  no  improvement  in 
quality  scores;   ten  percent  declined;  and  five  percent  improved.     For  those 
centers  in  which  the  scores  remained  unchanged,   the  same  items  received  low 
scores  in  both  assessments.  [35] 
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Some  outcome  oriented  assessment  programs  have  been  developed  (see  the 
preceding  section  in  this  chapter  and  also  Chapter  3).     Gonnella,  Louis,  and 
McCord  have  developed  the  "staging"  concept,  which  assumes  that  a  patient's 
status  at  a  particular  point  in  time  is  a  reflection  of  the  quality  of  the 
care  received  earlier.   [36]     They  have  divided  selected  diseases  into  stages 
to  reflect  the  progression  or  severity  of  the  condition.     The  technique  has 
been  applied  to  hospital  admissions  to  compare  the  quality  of  ambulatory  care 
before  admission  for  patient  populations  with  different  kinds  of  insurance 
coverage.     It  may  also  be  used  for  a  single  group  of  patients  to  help  identify 
deficiencies  in  prior  care  and  to  assess  changes  at  various  points  in  time. 
More  recent  applications  have  collected  information  at  the  ambulatory  level, 
using  both  chart  and  encounter  form  data.     Although  most  applications  have 
been  in  organized  health  care  programs,   the  investigators  believe  it  could 
also  be  used  in  small  offices.  [37] 

Most  ambulatory  care  is  delivered  in  small  independent  offices,   in  which 
there  are  very  few  quality  assurance  activities  under  way.     Some  special 
studies  have  been  conducted.     Peterson's  study  of  general  practice  in  North 
Carolina  is  noteworthy  for  employing  direct  observation  of  physicians  at 
work.    [38]     Jungfer  replicated  Peterson's  work  in  Australia,    [39]  and  Clute 
conducted  a  similar  study  in  Canada.    [40]     With  these  exceptions,  direct 
observation  has  seldom  been  used  to  assess  quality,   even  though  it  provides 
an  opportunity  to  evaluate  basic  skills  such  as  history  taking  and  physical 
examination,  which  cannot  be  assessed  from  the  medical  record. 

Kroeger's  study  of  the  office  practice  and  related  professional  activities 
of  internists  in  New  York  state  showed  variations  in  practice  patterns, 
particularly  in  the  use  of  laboratory  procedures.    [41]     About  two-thirds  of 
the  physicians  kept  sufficiently  accurate  records  to  be  included  in  the 
study.     The  researchers  concluded  that  it  is  possible  for  non-physicians 
to  abstract  office  records  which  can  be  used  later  by  physicians  to  assess 
the  quality  of  care. 

More  recently,   the  American  Society  of  Internal  Medicine  (ASIM)  has  developed 
criteria  for  assessing  the  quality  of  office  practice  and  tested  the  feasi- 
bility of  applying  those  criteria  in  evaluating  the  quality  of  care.  [42] 
Internists  from  six  geographical  areas  volunteered  to  participate  in  the 
study.     The  evaluation  was  based  on  special  encounter  forms  completed  by 
the  physician.     The  findings  show  a  high  level  of  agreement  among  the  geo- 
graphic areas  on  the  relative  importance  of  the  criteria,   but  a  wide  varia- 
tion among  regions  in  complying  with  criteria  items.     Response  rates  were 
very  low.     The  ASIM  suggests  that  these  scores  could  have  a  role  in  the 
development  of  regional  ranges  of  acceptable  performance  against  which  the 
practice  of  individual  physicians  could  be  measured,  but  the  technique  needs 
refinement . 

With  the  exception  of  these  studies,  most  ambulatory  quality  assessment  of 
small,   independent  office  practice  is  part  of  the  claims  review  process  for 
two  reasons:     the  claim  form  must  be  submitted  by  the  physicians  to  be  reim- 
bursed,  so  failure  to  complete  it  is  unlikely;   and  the  claims  review  process 
is  perhaps  the  only  point  of  leverage  over  solo  practice  physicians,  with 
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the  possible  exception  of  the  very  few  who  may  be  referred  to  medical 
societies  or  licensing  authorities  for  review  of  particularly  egregious 
behavior . 

Since  claims  review  systems  were  reviewed  in  Chapters  3  and  4,   they  will  not 
be  considered  here  in  detail,   except  to  emphasize  again  the  fact  that  they 
capture  very  limited  data.     Claims  review  systems  can  assess  the  quality  of 
care  by  uncovering  unnecessary  injections  and  inappropropr iate  prescriptions. 
They  may  generate  provider  and  patient  profiles  to  detect  over-utilization  of 
services  and  excessive  charges.     They  may  be  useful  in  monitoring  speciality 
consultations,  which  are  essentially  episodic.     Even  in  these  instances,  how- 
ever, more  refined  sampling  techniques  are  needed  to  improve  the  efficiency 
of  review. 

Claims  review  is  not  appropriate  for  monitoring  the  continuing  and  coordina- 
ting functions  of  primary  care,  unless  methods  are  devised  to  enrich  the 
data  base  and  provide  continuity  over  time. 

All  systems  and  approaches  to  quality  assurance  described  above  have  been 
inadequately  evaluated.  In  the  future,  it  will  be  important  to  determine 
the  extent  to  which  they  encourage  permanent  change  in  practice  patterns. 

STEERING  COMMITTEE  RECOMMENDATIONS 

1.  Ambulatory  claims  review  should  be  more  widely  implemented  in  an  experi- 
mental manner  while  more  appropriate  ambulatory  quality  assurance  techniques 
are  being  developed.     Despite  the  limitations  of  claims  review,   it  will 
permit  the  detection  of  the  most  serious  deficiencies.     Government  agencies 
and  other  purchasers  of  health  care  should  be  encouraged  to  require  more 
stringent  claims  review  by  their  fiscal  intermediaries.     Careful  evaluation 
of  these  programs  should  be  required. 

2.  Probability  sampling  techniques  should  be  developed  to  focus  on  patients 
and  providers  who  fall  at  the  extremes  of  distributions  of  care  patterns 
and  to  provide  estimates  of  the  broader  spectrum  of  care  provided  to  the 
total  population. 

3.  Closer  monitoring  is  required  of  pharmacy  services,  small  clinical  labora 
tories,  and  freestanding  radiological  units.  Assessment  techniques  using  pre 
identified  specimens  and  X-rays  should  be  more  systematically  applied  to  dete 
mine  the  accuracy  of  judgments  within  laboratories  and  radiology  services. 

4.  Methods  for  supplementing  information  captured  on  the  claim  form  should 
be  developed.     The  Minimum  Ambulatory  Care  Data  Set  [43]   (see  Appendix  E) 
should  be  the  basis  on  which  such  work  proceeds.     It  is  being  tested  and 
the  steering  committee  believes  it  should  be  more  widely  implemented.  In 
addition,  the  value  of  diagnostic,  patient,   and  laboratory  registries  to 
facilitate  problem  identification  and  provide  information  over  time  should 
be  explored. 
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5.  Quality  assurance  procedures  for  primary  ambulatory  care  should  be 
different  from  those  for  secondary  and  tertiary  care  and  should  concentrate 
on  the  unique  aspects  of  primary  care.     Many  ambulatory  quality  assessment 
programs  rely  on  a  diagnostic-specific  review  of  the  medical  record.  Since 
most  primary  ambulatory  care  consists  of  signs  and  symptoms  which  cannot 
readily  be  assigned  to  diagnostic  categories,  other  assessment  mechanisms 
are  needed.     Some  classification  schemes  which  incorporate  symptoms  languages 
are  already  being  developed  [44]  and  could  form  the  basis  of  an  experimental 
quality  assurance  project.     Another  approach  might  focus  on  the  basic  skills 
or  tasks  which  constitute  primary  ambulatory  care,  such  as  the  elicitation  of 
signs  and  symptoms  and  their  history,  performance  of  a  physical  exam,  the 
synthesis  of  this  information  into  recommendations  for  care,  and  determina- 
tion of  the  appropriate  point  for  referral.     Techniques  appropriate  for  re- 
viewing acute,  self-limiting  conditions  are  probably  inappropriate  for  moni- 
toring continuing  care  of  the  chronically  ill.     Information  to  reflect  the 
extent  to  which  the  practitioner  coordinates  care  provided  over  a  relatively 
long  period  of  time  is  seldom  found  in  the  medical  record,   and  other  re- 
cording and  assessment  methods  must  be  devised. 

6.  Additional  research  is  needed  to  document  current  patterns  of  ambulatory 
care,  giving  special  attention  to  unusual  methods  of  treatment  and  the  op- 
posite extremes  of  under  and  over-utilization.     The  reasons  for  such  varia- 
tions and  their  influence  on  patient  outcome  should  be  determined  and  con- 
sidered in  developing  standards  for  care. 

7.  Policy  mandates  with  respect  to  quality  assurance  should  impose  compar- 
able levels  of  stringency  on  all  health  care  delivery  arrangements,  even 
though  the  manner  in  which  requirements  are  met  may  vary.     The  greater  ease 
of  conducting  quality  assessment  activities  in  larger,   formally  organized 
health  care  programs,  such  as  Health  Maintenance  Organizations,  should  not 
lead  to  the  imposition  of  more  rigorous  requirements  on  such  organizations. 

8.  The  ambulatory  demonstration  projects  to  be  funded  by  the  Bureau  of 
Quality  Assurance  should  assist   in  developing  more  effective  methods  for 
ambulatory  quality  assurance.     The  widest  possible  range  of  approaches  should 
be  supported,  and  recipients  of  awards  should  not  be  limited  to  PSROs. 

9.  A  single  approach  to  quality  assurance  will  not  accommodate  the  diver- 
sity of  functions  and  personnel  included  within  the  ambulatory  care  sector. 
Further  research  is  required  before  a  range  of  proven  alternative  methods 
is  available. 


QUALITY  ASSURANCE  FOR  LONG-TERM  CARE 

Long-term  care  in  its  broadest  sense  is  both  a  treatment  situation  and  a 
living  situation.     It  encompasses  those  health  care  and  other  supportive 
services  provided  to  individuals  with  chronic  conditions  or  disabilities,  as 
well  as  the  environments  in  which  they  live.     It  is  fundamentally  different 
from  acute  hospital  care,   and  these  differences  must  be  considered  in  de- 
signing quality  assurance  programs. 
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Many  organizations  provide  long-term  care,   including  chronic  disease  hospi- 
tals,  extended  care  facilities,   specialized  rehabilitation  facilities, 
group  homes  or  domiciliary  care  facilities,  home  health  agencies,  day  care 
centers,   nursing  homes,  outpatient  clinics,   and  physicians'  offices.  However, 
an  accurate  count  of  the  number  of  providers  and  even  the  number  of  patients 
is  not  available  because  of  the  lack  of  uniform  definitions  and  data  reporting 
requirements.     The  average  length  of  stay  for  a  nursing  home  patient  is  about 
two  years,    [45]  which  emphasizes  the  importance  of  the  social  and  environ- 
mental,  as  well  as  physiological,   aspects  of  care. 

Patients  receiving  long-term  care  can  be  divided  into  two  major  groups:  those 
who  require  long-term  care  to  achieve  major  rehabilitation  goals  (usually 
people  recuperating  from  severe  accidents  and  illnesses  for  whom  care  may 
either  prevent  further  deterioration  but  bring  little  improvement,  or  produce 
significant  improvement  and  render  them  more  independent);   and  those  for  whom 
no  substantial  improvement   in  outcome  can  be  expected,  but  who  need  care  to 
be  maintained  at  their  present  level  of  function  for  as  long  as  possible  be- 
fore eventual  deterioration  and  death. 

People  in  the  second  category,   in  particular,  may  have  little  personal  lever- 
age to  exert  to  ensure  that  they  receive  quality  care.  For  the  most  part, 
they  are  old,    [46]   sick,    [47]   socially  isolated,    [48]   poor,    [49]  highly  medi- 
cated [50]  and  usually  depressed.  The  long-term  care  environment  may  be  their 
only  environment  for  the  duration  of  their  stay,  which  is  often  the  remainder 
of  their  lives.     Deficiencies  in  medical  or  nursing  care  or  in  housekeeping 
or  dietary  services,  which  perhaps  could  be  tolerated  during  a  brief  hospital 
stay,  become  intolerable  and  harmful  when  they  are  part  of  one's  daily  exis- 
tence for  years.  Thus,   structural  factors,   as  well  as  the  process  and  outcome 
of  care,   are  particularly  important  in  long-term  care  quality  assurance. 

In  designing  appropriate  ways  to  assure  quality  for  such  patients,   an  under- 
standing of  the  nature  of  chronic  illness  is  vital.     The  etiology  and  patho- 
genesis of  many  chronic  conditions  remain  obscure,   and  many  patients,  partic- 
ularly the  aged,  may  have  several  chronic  conditions.     Thus,   it  is  often 
impossible  to  categorize  patients  by  one  aspect  of  a  particular  disease 
process  and  develop  standards  and  criteria  for  judging  the  adequacy  of  their 
care.     Furthermore,  because  of  the  long-term  nature  of  the  conditions  and 
the  frequent  fluctuations  in  physical  and  mental  conditions,  treatment 
requirements  vary.     Patients  may  require  differing  levels  of  care  within  a 
relatively  short  time,   ranging  from  intensive  hospital  care,   skilled  nursing 
services,  custodial  care,  or  home  health  services,   to  periodic  office  visits. 
Methods  for  assessing  the  quality  of  care  rendered  to  such  patients  should 
include  all  sources  of  care  and  should  consider  the  impact  of  care  on  the 
patient's  expected  and  actual  ability  to  function  in  daily  life. 

Several  instruments  have  been  developed  to  assist  in  classifying  patients 
according  to  functional  abilities.     They  could  be  used  in  any  setting  to 
assess  appropriateness  of  placement,   as  well  as  the  actual  rather  than  expected 
progression  of  the  patient  throughout  the  course  of  illness.     Some  are  rela- 
tively simple  and  deal  essentially  with  socio-biological  functioning.  [51] 
They  include  such  activities  as  feeding  oneself,  continence,  transfer, 
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toileting,  dressing,  bathing,   and  walking.     Others  are  broader  and  include 
special  needs  such  as  medication,  diet,  or  safety  supervision,  as  well  as 
mental  status  and  ability  to  cope  with  social  situations.  [52] 

The  Collaborative  Patient  Assessment  Instrument  (CPAI)  was  developed  jointly 
by  four  university  research  groups  and  is  a  multipurpose  data  set  which  would 
permit  a  variety  of  assessments.    [53]     It  contains  information  about  patients' 
sociodemographic  characteristics,   functional  status,   impairment,  medical 
status  (including  risk  factor  measurements  such  as  smoking),  and  medically 
defined  conditions.     More  recent  applications  have  included  items  on  services, 
medications,  and  tests  received  by  patients. 

Use  of  a  24-item  minimum  basic  data  set  was  recommended  by  the  Conference  on 
Long-Term  Health  Care  Data.     The  set  contains  demographic  data,  information 
about  the  health  and  functional  status  of  the  patient,  provider  character- 
istics, and  type  of  service  rendered.    [54]     Some  items  can  be  implemented 
immediately;  others  need  further  refinement.     The  total  data  set  will  pro- 
vide uniform  information  to  assist  in  health  planning  and  policy  decisions, 
the  management  of  individual  facilities,  the  monitoring  of  patient  care,  and 
related  research. 

Some  assessments  of  the  reliability,  validity,   and  costs  of  these  instruments 
have  been  conducted.     In  general,   the  simpler  instruments  are  more  reliable 
and  less  expensive.     The  more  comprehensive  instruments,  which  include  an 
assessment  of  mental  status  and  social  functioning,  are  less  reliable  and  may 
require  more  highly  trained  personnel  to  administer  them.     Nonetheless,  the 
instruments  are  available.     They  could  be  applied  in  a  quality  assurance  con- 
text, but  few  have  been  used  in  that  manner. 

As  an  alternative  to  measuring  functional  status  over  time,  some  assessments 
have  focused  on  specific  occurrences  as  indicators  of  the  overall  quality  of 
care  within  a  facility.     Examples  include  assessments  of  the  prevalence  of 
decubitus  ulcers,   the  incidence  of  urinary  tract  infections  in  catheterized 
patients,  or  the  use  of  particular  drugs.   [55]     Although  such  investigations 
may  be  helpful  in  identifying  facilities  which  need  closer  review,  they  do 
not  constitute  ongoing  quality  assurance  systems  as  defined  in  this  study. 

A  few  special  research  projects  have  included  comprehensive  reviews  of  the 
quality  of  long-term  care.     A  study  of  nursing  homes  in  the  Denver  area 
identified  deficiencies  in  medical  record  documentation,  the  value  of 
diagnosis  as  a  predictor  of  patient  needs,  and  the  present  facility  classifi- 
cation system.     The  need  for  a  reimbursement  system  based  on  functional  sta- 
tus and  service  needs  was  noted.   [56]     Batelle  Institute  is  assessing  the 
quality  of  long-term  care  by  studying  mental  functional  status,  physical 
functional  status,   satisfaction,   and  morale.     The  relation  between  structure, 
process,   and  outcome  measures  of  quality  will  be  examined.    [57]  Williams 
is  assessing  the  effectiveness  of  two  kinds  of  patient  evaluation  and  place- 
ment techniques:     direct  evaluation  by  a  physician  and  public  health  nurse 
and  evaluation  on  the  basis  of  written  reports.     Changes  in  health  and  func- 
tional status  of  both  institutionalized  and  non-institutionalized  patients 
will  be  examined.     The  costs  of  alternative  long-term  care  settings,  including 
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home  care,  will  be  explored.  [58] 

Some  quality  assurance  systems  have  been  developed.     The  JCAH  Long-Term  Care 
Audit  System  [59]   is  similar  to  a  system  proposed  by  Ainsworth  and  Boyce,  [60] 
but  neither  appears  to  be  widely  used.     The  Hospital  Utilization  Project  (HUP) 
in  Pittsburg  sponsors  an  abstracting  system  for  skilled  nursing  facilities 
which  can  be  used  for  both  utilization  review  and  medical  care  evaluation 
studies.    [61]     The  comprehensive  Quality  Evaluation  System  developed  by  re- 
searchers at  Rush  Presbyterian-St .  Luke's  Medical  Center  includes  a  survey  of 
both  facilities  and  patients.     It  is  being  tested  for  routine  use  by  the 
Illinois  Department  of  Health.   [62]     The  EMCRO  experience  in  nursing  home  re- 
view has  been  summarized  by  Arthur  D.  Little,   Inc.  [63] 

PSRO  guidelines  for  long-term  care  review  have  only  recently  been  considered 
by  the  National  Professional  Standards  Review  Council  and  have  not  been 
implemented.     They  appear  to  specify  a  model  similar  to  the  PSRO  hospital 
review  of  acute  care,  with  a  few  exceptions.   [64]     A  multidisciplinary  group 
of  providers  is  to  be  involved  in  all  phases  of  long-term  care  review,  which 
is  a  departure  from  the  physician  domination  of  acute  care  review.  Pre- 
admission certification  is  required.     Additional  requirements  are  an  examina- 
tion of  the  extent  to  which  services  provided  meet  the  individual  patient's 
needs  and  at  least  one  bedside  review  of  the  patient  each  year.     Whether  these 
review  requirements  are  adequate  and  appropriate  to  ensure  quality  for  long- 
term  care  should  be  carefully  evaluated. 

The  quality  of  non-institutional  care  for  the  chronically  ill  has  not  been 
extensively  considered  except  in  the  context  of  more  general  quality  assur- 
ance programs  for  ambulatory  care.     Katz  has  studied  patients  released  from 
a  chronic  disease  rehabilitation  hospital  to  determine  whether  home-bound 
patients  are  more  likely  to  maintain  or  increase  physical,  psychological, 
and  social  function  with  or  without  the  supervision  of  a  public  health 
nurse.   [65]     HUP  formerly  sponsored  an  abstracting  system  for  home  health 
agencies  which  has  been  discontinued — reportedly  because  of  the  lack  of 
comparable  data.    [66]     More  recently,   the  National  League  for  Nursing  is 
revising  a  statistical  reporting  manual  for  public  health  nurse,  which  is 
similar  to  a  minimum  data  set.    [67]     Interest  in  assessing  the  quality  of 
home  health  services  is  growing. 

Except  for  the  Medicare  and  Medicaid  utilization  review  requirements,  the 
primary  continuing  review  of  long-term  care  is  the  structural  assessment 
which  is  part  of  the  facility  licensing  and  certification  process  and  is 
assumed  to  influence  the  safety  and  quality  of  patient  care.     Examples  of 
items  for  assessment  include  the  presence  of  automatic  sprinkler  protec- 
tion against  fires,  precautions  so  that  doors  to  hazardous  areas  are  not 
held  open  automatically,  and  specifications  about  the  numbers,  training, 
and  supervision  of  personnel.     These  requirements  are  limited  and  evidence 
shows  that  they  are  not  uniformly  enforced. 

A  GAO  study  concluded  that  despite  required  ratios  of  staff  to  patients 
and  inspection  programs  to  enforce  them,  numerous  nursing  homes  in  the 
study  sample  violated  these  requirements.   [68]     Failure  to  meet  the 
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required  number  of  physician  visits  has  been  documented.    [69]     A  DHEW  report 
indicated  failure  to  comply  with  fire  safety  regulations  and  concluded  that 
only  6.1  percent  of  the  long-term  care  facilities  in  a  national  sample  met 
all  requirements  of  the  1967  Life  Safety  Code.    [70]     Errors  in  prescribing 
and  administering  drugs  have  been  documented.    [71]     There  are  shortages  of 
therapy  personnel  and  services.   [72]     Most  nursing  care  is  provided  by  un- 
skilled, unlicensed,  and  low  paid  personnel.   [73]     The  high  turnover  rate  of 
staff  is  also  a  problem. 

The  importance  of  government  financing  in  the  provision  of  long-term  care 
services  is  increasing.     Medicaid  supports  over  50  percent  of  the  nation's 
nursing  home  bill.    [75]     Reimbursement  programs  could  provide  a  lever  to 
improve  both  the  adequacy  of  facilities  and  the  quality  of  care.     Part  of 
the  reason  this  has  not  been  done,  however,  may  be  that  strict  enforcement 
of  facility  standards  would  mean  the  closing  of  some  marginal  facilities. 
There  may  be  no  other  source  of  care  for  the  residents  of  those  facilities, 
and  thus,   inspectors  are  reluctant  to  close  them. 

Current  reimbursement  mechanisms  contribute  to  inappropriate  care.  By 
requiring  certain  progressions  of  care  (requiring  a  hospital  stay  in  order 
to  be  eligible  for  lower  levels)  and  designing  the  reimbursement  to  match 
the  facility  rather  than  the  patient,  reimbursement  mechanisms  destroy  the 
continuity  of  care  and  lower  the  quality  of  life. 

The  difficulties  in  providing  long-term  care  of  high  quality  should  not  imply 
that  all  facilities  fail.     Indeed,  many  innovative  centers  serve  as  models 
which  should  be  emulated.     However,  the  current  status  of  long-term  care  is 
described  by  documented  examples  of  inferior  quality,   the  failure  to  apply 
appropriate  existing  quality  assessment  techniques,   the  application  of  assess- 
ment techniques  which  may  be  inappropriate  or  inadequately  enforced,  reim- 
bursement mechanisms  which  contribute  to  these  problems,  and  the  absence  of 
funding  to  bring  about  improvement  in  the  near  future. 


STEERING  COMMITTEE  RECOMMENDATIONS 

r.     Quality  assurance  programs  for  long-term  care  should  be  designed  to 
address  the  unique  needs  of  the  chronically  ill.     The  etiology  of  many 
chronic  conditions  remains  obscure,  and  many  individuals,   particularly  the 
aged,  have  several  chronic  conditions.     An  assessment  of  quality  based  on 
diagnostic-specific  criteria  is  often  inappropriate,  and  functional  status 
is  a  more  relevant  measure.     Furthermore,  because  of  the  long-term  nature 
of  the  patient's  condition  and  frequent  fluctuations   in  physical  and  mental 
states,   treatment  requirements  vary.     Patients  may  require  differing  levels 
of  care  within  a  short  time,  ranging  from  intensive  hospital  care,  skilled 
nursing  services,  custodial  care,  or  home  health  services,   to  periodic  office 
visits.     Methods  for  assessing  the  quality  of  care  rendered  to  such  patients 
should  include  all  sources  of  care  and  should  consider  the  effect  of  care 
on  the  patient's  expected  and  actual  ability  to  function  in  daily  life. 
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2.  The  responsibility  for  quality  assurance  in  long-term  care  belongs  at 
the  community  level.     Anything  less  will  be  based  on  evaluation  of  care  from 
the  fragmented  view  of  individual  facilities  or  programs  and  will  perpetuate 
the  inefficient  and  costly  services  which  currently  exist.     Steps  should  be 
taken  to  develop  community-level  organizations  to  include  a  broad  range  of 
providers,  facilities,  professional  groups,  consumers,  and  representatives 
from  planning  and  certifying  agencies.     The  community  organization  should 
consider  such  issues  as  access  to  care,  appropriateness  of  placement,  scope 
of  available  services,   the  structural  characteristics  of  services,  and  the 
accumulation  of  uniform  data  to  both  plan  for  and  assess  long-term  care 
services.     Assessment  of  the  technical  components  of  care  could  be  delega- 
ted to  PSROs  and  other  groups  of  health  care  providers. 

3.  Demonstrations  to  test  the  feasibility  of  a  community  approach  in  terms  of 
both  cost  and  effectiveness  should  be  initiated.     Evaluations  should  occur 
after  prototype  organizations  have  passed  the  developmental  phase. 

4.  The  uniform  data  elements  and  common  language  for  reporting  on  long-term 
care  services  proposed  by  the  Conference  on  Long-Term  Health  Care  Data  should 
be  required  nationally  (see  Appendix  E).    [76]     Most  items  can  be  implemented 
immediately,  but  further  work,  is  required  to  define  mental  functioning, 
social  functioning,  certain  aspects  of  physical  functioning,  and  the  events 
or  reasons  for  use  of  services  other  than  diagnoses.  These  data  will  not  only 
facilitate  the  development  of  long-term  care  quality  assurance,  but  will 
assist  in  patient  care,  program  planning  and  evaluation,  and  policymaking. 

5.  The  data  set  should  be  required  in  all  records  and  reporting  systems  of 
programs  serving  long-term  care  patients  under  Titles  18,  19,  and  20  of  the 
Social  Security  Act.     Patient  assessment  based  on  these  data  should  be  re- 
quired at  successive  intervals. 

6.  State  and  federal  reimbursement  policies  for  long-term  care  should  be  re- 
formed.    State  and  federal  regulations  for  reimbursement  and  accounting  should 
be  made  compatible  and  redesigned  to  enhance  their  influence  on  the  quality 

of  care.     The  levels  of  reimbursement  should  not  be  so  inadequate  as  to  lead 
to  poor  quality.     Topics  for  experimental  reimbursement  projects  might  in- 
clude capitation,  which  would  permit  the  individual  to  move  from  one  level  of 
care  to  another  without  being  penalized.     Another  option  is  to  establish  the 
reimbursement  rate  for  a  specific  facility  on  the  basis  of  its  usual  mix  of 
patients  and  then  permit  patients  to  be  moved  from  one  level  to  another  de- 
pending on  their  conditions. 

7.  The  certification  and  licensure  process  for  long-term  care  providers  should 
be  reconsidered.     The  Department  of  Health,  Education,   and  Welfare  study, 
scheduled  to  begin  in  April  of  1977,   should  go  beyond  a  review  of  existing 
structural  standards  to  address  more  fundamental  issues  of  quality  and  analyze 
the  financial  and  other  ramifications  of  forced  compliance  with  standards. 

8.  Support  of  existing  programs  to  train  personnel  to  work  in  long-term  care 
should  be  continued  and  expanded.     Program  content  should  focus  on  the  unique 
characteristics  of  long-term  care,  the  multiplity  of  skills  required  to  meet 
patient  needs,   the  necessity  of  a  team  approach,   and  an  appreciation  of  the 
contributions  of  all  health  professionals  in  providing  high  quality  care. 
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9.  The  long-term  care  quality  assurance  demonstrations  to  be  funded  by  the 
Bureau  of  Quality  Assurance  should  include  a  variety  of  alternative  approaches 
to  review  and  should  not  be  limited  to  PSROs.     More  basic  research  is  also 
required . 

10.  Existing  standards  to  protect  the  residents  of  long-term  care  facilities 
should  be  enforced  while  improved  mechanisms  for  assessing  the  quality  of 
long-term  care  are  evolving. 

QUALITY  ASSURANCE  AND  IMPLICATIONS  FOR  PROVIDER  PERFORMANCE 

Health  care  quality  assurance  programs  are  generally  based  on  two  assumptions: 
first,  that  quality  can  be  defined  and  measured;  and  second,  that  when  inap- 
propriate care  is  detected,  specific  actions  can  be  taken  to  correct  the  de- 
ficiency and  ultimately  improve  the  quality  of  care.     The  experiences  of 
operating  quality  assurance  programs  lead  one  to  question  these  assumptions, 
however . 

Correcting  inadequacies  in  the  patient  care  process  is  one  of  the  major  dif- 
ficulties encountered  by  review  programs,  particularly  when  deficiencies  re- 
late to  the  clinical  decision-making  process,  rather  than  simply  a  decision 
to  admit  or  discharge  a  patient  from  the  hospital. 

In  part,  the  difficulty  may  reflect  inadequate  information  about  the  reason 
for  deficiencies.     There  are  some  data  from  medical  audits  to  suggest  that 
many  deficits  in  patient  care  do  not  stem  from  lack  of  knowledge  and,  thus, 
cannot  be  remedied  by  traditional  education  techniques.   [77]     Some  studies 
have  revealed  inadequate  basic  clinical  skills.   [78]    Additional  information 
shows  failure  to  conform  with  explicit  criteria  for  treating  patients  with 
particular  diagnoses.   [79]     In  general,  however,  current  patterns  of  practice 
are  not  well  enough  documented  to  permit  the  design  of  immediately  relevant 
corrective  measures. 

The  situation  is  further  complicated  by  the  professional  tradition  of  medi- 
cine, which  emphasizes  "self-government  for  the  profession  as  a  whole  and 
autonomy  for  each  practitioner  within  the  limits  laid  down  by  the  profes- 
sion."  [80]     The  traditional  reluctance  to  pass  judgment  on  one's  peers  has 
been  partially  superceded  by  recent  demands  for  professional  accountability. 
Nevertheless,  the  effectiveness  of  attempts  at  change  is  primarily  governed 
by  the  visibility  of  support  within  the  professional  community  and  the  amount 
of  peer  pressure  which  is  brought  to  bear  to  achieve  conformity. 

These  difficulties  notwithstanding,   several  approaches  have  been  (or  could 
be)  used  in  an  attempt  to  change  provider  performance  and  improve  the  quality 
of  care. 

Programs  of  continuing  medical  education  (CME)  are  proliferating  as  CME  is 
increasingly  being  required  as  a  condition  of  membership  in  state  medical 
associations  and  specialty  societies,  as  a  legislative  requirement  for  re- 
licensure,   and  in  order  for  hospitals  to  receive  JCAH  accreditation.  The 
educational  supplement  to  the  Journal  of  the  American  Medical  Association 
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lists  4,862  approved  CME  programs.     A  rough  estimate  of  total  annual 
national  expenditures  for  CME  approximates  $1,850,000,000.  [81] 

In  some  instances  physicians  have  improved  their  medical  practice  after 
attending  CME  programs.     For  example,  the  Back-to-Medical  School  program 
at  Mills  Hospital  in  San  Mateo,  California  has  positively  influenced  the 
performance  of  staff  physicians  from  hospitals  in  that  area.   [82]     The  pro- 
gram consists  of  two  one-hour  sessions  per  week  devoted  to  basic  clinical 
skills,  recent  clinical  advances,  and  appropriate  indications  for  medical 
procedures.     The  performance  of  specific  procedures  was  assessed  one  year 
before  and  after  relevant  educational  sessions,  and  definite  improvements 
were  shown. 

The  effect  of  an  intensive  course  in  cardiac  auscultation,  which  consisted 
of  12  to  20  hours  of  presentations  over  a  two  to  three-day  period,  has  also 
been  assessed.   [83]    Although  the  ability  to  recognize  accurately  normal  and 
abnormal  heart  sounds  improved  immediately  after  the  course  for  most  partici- 
pants, measurements  taken  six  months  later  were  not  significantly  different 
from  those  taken  prior  to  the  course. 

In  most  cases,  the  effect  of  CME  has  not  been  assessed.     A  review  of  the 
educational  supplement  suggests  that  most  courses  are  oriented  toward  speci- 
fic diseases  or  conditions  and  presented  in  a  manner  which  resembles  the 
medical  school  curriculum.     The  relevance  of  undergraduate  medical  training 
to  continuing  education  of  a  mature  professional  has  been  questioned.  [84] 
The  courses  rarely  relate  to  basic  skills  in  the  daily  practice  of  medicine, 
such  as  history  taking  or  physical  examinations,  effectiveness  of  treatment, 
or  quality  assessment.     Furthermore,   there  is  some  evidence  of  a  lack  of 
significant  correlation  between  the  number  of  courses  attended  and  the 
quality  of  physician  performance.   [85]  In  general,   the  effectiveness  of 
traditional  continuing  medical  education  in  improving  the  quality  of 
patient  care  has  not  been  convincingly  demonstrated. 

Some  alternative  educational  approaches  attempt  to  provide  greater  relevance 
for  the  practitioner's  daily  activities  by  addressing  identified  performance 
deficits  (based  on  either  group  or  individual  data)  or  by  attempting  to 
simulate  the  routine  practice  of  medicine. 

Williamson's   [86]  and  Brown's   [87]  early  efforts,  which  link  medical  staff 
education  programs  with  performance  deficits  and  require  a  subsequent  re- 
audit  to  determine  the  level  of  improvement,  were  described  in  Chapter  3. 
These  concepts  have  been  incorporated  in  medical  audit  requirements  by  PSROs , 
the  JCAH,  and  the  California  Medical  Association.     But  achieving  long  lasting 
improvements  remains  difficult.     Sivertson  et  al.   in  Wisconsin  have  designed 
individual  physician  profiles,  which  provide  the  basis  for  professional 
consultation  and  the  development  of  an  educational  program  to  meet  each 
physician's  needs.   [88]     Hamaty's  program  in  West  Virginia  was  also  tailored 
to  the  practice  of  individual  physicians.   [89]    Anecdotal  improvements  have 
been  noted  in  both  situations,  but  systematic  evaluations  are  lacking. 
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Educational  techniques  which  simulate  the  patient  care  process  are  useful 
because  of  their  flexibility  and  interactive  nature.     Both  computerized  and 
paper-and-penc il  simulations  are  being  developed,    [90]  but  their  implementa- 
tion in  quality  assurance  programs  is  currently  limited. 

As  an  alternative  to  formal  education  programs,  much  medical  learning  occurs 
informally  as  a  by-product  of  routine  practice.     Conversations  in  hospital 
dining  rooms,  discussions  of  problem  cases  with  colleagues,  and  consulta- 
tions may  all  help  to  upgrade  a  physician's  performance.     Some  medical  care 
delivery  settings  capitalize  on  the  potential  of  informal  learning  by  develop- 
ing organizational  features  which  foster  these  informal  processes. 

Some  studies  indicate  that  organized  arrangements  whereby  physicians  practice 
in  groups  facilitate  the  exhange  of  professional  information,  guide  the  flow 
of  patients  among  professionals,  provide  access  to  a  wider  range  of  resources, 
and  result  in  more  appropriate  patient  care.   [91]  Interpretation  of  results 
is  complicated  by  the  fact  that  many  group  practices  are  also  reimbursed  on  a 
capitation  basis,  so  the  influence  of  organization  on  performance  is  difficult 
to  separate  from  the  influence  of  reimbursement.     Nevertheless,   there  are  dif- 
ferences among  prepaid  plans,    [92]  and  among  non-prepaid  practices,   [93]  which 
apparently  stem  from  organizational  characteristics. 

Hospital  characteristics  have  also  been  related  to  variations  in  the  use  of 
resources  and  quality  of  care,  but  the  findings  are  more  contradictory.  Size 
is  sometimes  associated  with  improved  care  [94]  and  sometimes,  with  poorer 
care.   [95]     Intervening  variables  apparently  are  key.     Structured  medical 
staffs  with  salaried  full-time  department  chiefs  are  associated  with  improved 
care.   [96]     Coordination  is  also  an  important  attribute  leading  to  increased 
efficiency  and  quality  of  care.    [97]     A  study  currently  under  way  hypothesizes 
that  working  relationships  at  the  ward  level  will  be  more  influential  than 
overall  hospital  characteristics.  [98] 

Technology  may  be  used  as  a  tool  for  improving  the  quality  of  care.  The 
Harvard  Community  Health  Plan  in  collaboration  with  the  Laboratory  of  Computer 
Science  of  Massachusetts  General  Hospital  has  developed  a  Computer-Stored 
Ambulatory  Record  (COSTAR) ,  which  is  an  information  and  communication  system 
replacing  the  traditional  paper  medical  record.     Standards  of  care  are  being 
developed  for  several  diagnoses  or  conditions  to  assist  in  patient  monitoring. 
Deviations  from  standards  are  identified  at  the  time  they  occur,   so  that  feed- 
back of  information  to  the  physician  can  be  both  relevant  and  timely.  One 
protocol  identifies  patients  with  positive  throat  cultures  for  beta  hemolytic 
streptococci  for  whom  appropriate  antibiotic  therapy  is  not  recorded  within 
four  days  of  treatment.     Seven  months  after  program  initiation,  only  three 
patients  did  not  begin  appropriate  therapy  within  ten  days  after  treatment. 
A  similar  program  for  patients  with  newly  discovered  hypertension  is  being 
evaluated.    [99]     The  system  also  has  a  follow-up  capacity  for  either  individ- 
uals or  groups  of  patients.     When  the  Food  and  Drug  Administration  removed  a 
sequential  birth  control  pill  from  the  market,   the  names  of  women  receiving 
this  medication  were  retrieved  by  computer.     Each  women  was  contacted  by  her 
physician  to  arrange  for  alternative  medication.  [100] 


103 


At  a  more  rudimentary  level,   technology  may  refer  to  the  basic  instruments, 
equipment,  and  support  services  in  the  office  or  hospital.     Some  inadequacies 
in  patient  care  are  caused  by  the  absence  of  such  items,  and  the  purchase  of 
equipment  or  administrative  changes  may  be  more  effective  than  education  pro- 
grams in  leading  to  improvements. 

The  influence  of  rewards  and  incentives  has  not  been  systematically  explored 
in  relation  to  quality  assurance,  although  certainly  they  are  influential  in 
the  practice  of  medicine.     The  current  reward  structure  does  not  recognize  the 
provision  of  high  quality  patient  care.     One  of  the  few  explicitly  identified 
rewards  for  practicing  physicians  is  the  Physician  Recognition  Award  (PRA), 
which  is  given  by  the  AMA  for  participating  in  continuing  education  activi- 
ties for  a  qualifying  period  of  three  years.   [101]     In  the  absence  of  evi- 
dence that  CME  influences  practice,  however,   it  is  difficult  to  assume  that 
receiving  the  PRA  is  an  indication  of  high  quality  care. 

Additional  incentives  for  improving  performance  may  be  provided  by  the  volun- 
tary self-assessment  programs  being  developed  by  specialty  societies  and 
boards  to  assist  in  recertif ication.     The  American  College  of  Physicians' 
Medical  Knowledge  Self-Assessment  Examination  can  be  purchased  by  physicians 
who  receive  a  syllabus,  practice  questions,  and  CME  courses.     After  taking  the 
self-assessment  exam  they  receive  test  scores  and  additional  summaries,  bib- 
liographies, and  reprints  for  further  preparation  before  the  recertif ication 
exam.     The  names  of  physicians  who  pass  the  recertif ication  exam  will  be 
noted  by  the  American  Board  of  Internal  Medicine  in  the  Directory  of  Medical 
Specialties.     The  recertif ication  process  for  the  American  Board  of  Family 
Practice  will  include  a  review  of  office  records  in  disease  categories  se- 
lected by  the  physician,  in  addition  to  a  cognitive  exam.  [102] 

Incentive  payments  have  been  used  by  the  British  National  Health  Service  to 
encourage  continuing  education,  as  well  as  specific  types  of  services  such  as 
home  visits,  pap  smears,  and  family  planning.   [103]     Comparable  examples  in 
this  country  are  difficult  to  identify. 

It  has  been  hypothesized  that  some  prepaid  group  health  plans  have  been  able 
to  reduce  their  use  of  hospitals  by  permitting  physicians  to  share  in  savings 
which  result  from  lower  hospital  utilization  rates.   [104]     In  order  for  the 
effectiveness  of  such  incentives  to  be  convincingly  documented,  however,  the 
impact  of  the  physician  bonus  would  have  to  be  isolated  from  all  other  factors 
which  might  be  associated  with  reduced  hospitalization  in  prepaid  health  plans. 
This  has  not  been  done. 

The  most  extreme  method  of  behavior  change  is  the  imposition  of  sanctions, 
but  even  here,  many  options  are  available.     Some  hospital  medical  staffs  and 
medical  groups  have  developed  internal  policies  which  specify  accepted  prac- 
tices and  sanctions  for  deviations,   including  limitation  of  practice  privi- 
leges, mandatory  consultation,  and  eventual  suspension  from  staff.     The  threat 
of  their  imposition  may  encourage  adherence  to  standards,  but  little  is  known 
about  the  effectiveness  of  such  policies. 

Denial  of  reimbursement  is  a  form  of  sanction.     A  decrease  in  unnecessary 
injections  in  both  New  Mexico  and  the  San  Joquin  area  was  observed  after 
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payments  were  denied  (see  Chapter  4).  It  has  been  suggested,  however,  that  phy 
sicians  may  copmpensate  by  increasing  the  use  of  procedures  which  they  know  are 
reimbursable,    [105]   so  total  change  in  inappropriate  behavior  may  be  minimal. 

Imposition  of  more  drastic  sanctions  such  as  restriction  of  privileges  or  re- 
vocation of  licenses  occurs  less  frequently — in  part,  perhaps,  because  they 
are  so  drastic.     Less  severe,   intermediate  sanctions  might  be  applied  more 
readily,   and  thereby  provide  more  effective  leverage  for  improving  inappro- 
priate practice. 

Despite  the  variety  of  potential  mechanisms  for  encouraging  improved  provider 
performance,   existing  experience  is  somewhat  discouraging.     Some  successes 
are  seen,  but  the  factors  associated  with  them  have  not  been  isolated  in  a 
manner  which  would  permit  more  general  application.     More  typically,  provider 
performance  did  not  change  or  the  change  was  not   long  lasting. 

STEERING  COMMITTEE  RECOMMENDATIONS 

1.  Quality  assurance  programs  should  not  mandate  any  particular  method  to 
improve  provider  performance,   including  continuing  medical  education.  Evi- 
dence of  effectiveness  is  inadequate. 

2.  Research  is  needed  to  categorize  deficiencies  in  the  quality  of  care,  de- 
termine the  reasons  for  deficiency,   and  design  appropriate  corrective  actions. 
All  methods  for  improvement  should  be  carefully  evaluated  to  determine  the 
extent  to  which  they  result  in  lasting  behavior  change. 

3.  Hypotheses  about  optimal  learning  conditions   in  other  fields  should  be 
further  explored  in  medical  settings.     The  impact  of  working  arrangements 
which  make  performance  more  visible  and  facilitate  the  sharing  of  information 
should  be  evaluated,   since  it  is  hypothesized  that  such  arrangements  also 
improve  the  quality  of  care.     The  potential  for  informal  learning  in  actual 
medical  practice  should  be  examined.     For  example,  medical  consultation  may 
provide  an  opprotunity  for  the  consultant  to  enrich  his  appreciation  for 
social-psychological  aspects  of  medical  practice  and  the  referring  physician 
to  upgrade  his  technical  skills.     The  point  at  which  consultation  is  sought 
then  becomes  important. 

4.  Additional  research  is  needed  on  the  relative  effects  of  alternative 
methods  for  providing  the  physician  with  assessment  results  indicating  need 
for  improvement.     These  might  include  simple  unstructured  feedback  of  infor- 
mation,  a  more  targeted  response  concentrating  on  particular  areas,   or  the 
provision  of  incentives  for  review  and  change.     The  reasons   for  failure  to 
change  should  be  explored. 

5.  Research  on  the  diffusion  of  medical  information  might  identify  potential 
points  of  intervention  to  hasten  the  process  of  information  exchange. 

6.  The  effect  of  sanctions  which  are  less  drastic  than  permanent  loss  of  li- 
censure should  be  tested  where  clearly  inappropriate  care  is  identified  and 
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behavior  does  not  change.     Experimentation  with  intermediate  sanctions  should 
include  temporary  curtailment  of  privileges,  licensing  with  restrictions  on 
specified  areas  of  practice,  mandatory  supervision  of  medical  practice  for  one 
year,  remedial  education,  or  curtailment  of  privileges  for  certain  procedures 
(perhaps  surgical)  which  are  seldom  performed.     Related  demonstrations  should 
test  the  effectiveness  of  equipping  PSROs  with  a  wider  range  of  sanctions, 
including  more  direct  links  with  licensing  bodies  or  perhaps  authorization 
to  remove  a  license  with  due  cause. 

7.  State  legislative  bodies  should  waive  or  amend  existing  statutes,   if  ne- 
cessary, to  permit  the  experimentation  suggested  above. 

8.  The  feasibility  and  effectiveness  of  publicizing  instances  of  persistently 
poor  quality  by  individual  practitioners  in  public  media  should  be  explored. 


QUALITY  ASSURANCE  AND  THE  HEALTH  CARE  CONSUMER 

Consumers  of  health  care  traditionally  have  not  been  involved  in  either  for- 
mally assessing  the  quality  of  care  or  reviewing  the  results  of  such  assess- 
ments in  a  manner  which  might  encourage  them  to  take  actions  beneficial  to 
their  own  health  or  influence  the  behavior  of  the  health  care  professional. 
Some  might  argue  that  this  role  is  inappropriate,  since  consumers,  or  pa- 
tients, are  inadequately  prepared  both  intellectually  and  emotionally  to  make 
such  assessments  or  to  act  rationally  upon  the  information.     As  one  might 
expect,  at  least  one  study  [106]  indicates  that  patients  are  currently  poor 
judges  of  the  technical  adequacy  of  their  medical  care.     However,  there  is 
also  some  evidence  to  the  contrary.     In  any  case,  this  is  only  one  dimension 
of  quality.     The  consumer's  attitudes  and  expectations  about  health  care, 
the  provider's  technical  and  supportive  services,   and  the  manner  in  which  the 
two  interrelate  all  influence  quality  as  measured  by  health  outcome. 

Consumers  already  informally  assess  the  medical  care  they  receive  and  take 
actions  accordingly,  either  personally  or  collectively.     As  the  accuracy  of 
their  perceptions  about  health  care  increases,  their  use  of  the  health  care 
system  should  become  more  appropriate.     The  challenge,   then,   is  to  find  ways 
of  involving  the  consumer  in  the  health  care  process,  educating  him  to  accept 
more  responsibility  for  his  own  health  and  use  resources  more  appropriately, 
while  simultaneously  drawing  upon  his  experiences  to  make  qualitative  assess- 
ments of  the  delivery  of  care  and  suggest  potential  alteration  where  war- 
ranted.     The  manner  in  which  this  can  be  accomplished  is  not  clear,  but  ini- 
tial beginnings  can  be  made. 

Several  levels  of  consumer  involvement  can  be  envisaged.     Increasing  federal 
support  of  health  care  programs  implies  a  responsibility  to  assure  the 
appropriateness  and  quality  of  services  purchased.     Health  care  consumers 
and  lay  policymakers,  as  well  as  health  professionals,  should  be  in  a  posi- 
tion to  express  opinions  and  to  have  a  voice  in  assessing  the  return  on  their 
investments.     Similar  decisions  about  the  use  of  resources  are  made  in  state 
planning  agencies,  group  health  plans,  and  hospital  boards  of  directors,  with 
a  subsequent  need  to  evaluate  the  results.     At  the  individual  level,   the  need 
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for  involvement  and  understanding  is  perhaps  even  more  compelling,   since  the 
individual's  health  status  is  influenced  by  his  behavior.     Some  would  argue 
that  changes  in  one's  environment  and  way  of  life,  rather  than  necessarily  re- 
ceiving more  and  bettter  health  care,   are  key  factors  in  improving  health.  [107] 

A  growing  body  of  literature  and  experience  is  emerging  which  deals  with  con- 
sumer involvement  and  behavior.     In  Green's  discussion  of  factors  influencing 
health  behavior,  he  describes  predisposing  factors  as  those  psychological  and 
social  forces,   such  as  attitudes  and  knowledge,  that  induce  people  to  act  or 
not  to  act  in  specific  ways.   [108]    Attitudes  toward  health  care  (especially 
satisfaction)  and  health  information  or  education  have  been  related  to  health 
behavior  and  outcome.     They  suggest  that  some  specific  changes   in  the  provision 
of  care  may  improve  the  results. 

Studies  of  patient  satisfaction  without  reference  to  particular  health  care 
encounters  are  not  particularly  helpful  for  quality  assessment,  except  to  note 
the  demographic  and  utilization  characteristics  of  patients  who  are  more  or 
less  satisfied  with  health  care  in  general.    [109]     Studies  of  patient  satisfac- 
tion with  respect  to  particular  practice  arrangements  yield  more  specific 
findings.     Patients  can  adapt  to,  and  be  satisfied  with,  changes  in  the  manner 
in  which  care  is  provided — in  particular,   to  increased  use  of  paramedical 
workers.    [110]     Satisfaction  with  prepaid  group  practice  increases  as  patients 
gain  experience  in  using  the  services,  acquire  more  information  about  the  plan, 
and  become  generally  more  familiar  with  its  provisions.    [Ill]     But  as  overall 
satisfaction  increases  and  patients  become  more  involved  with  a  particular 
health  care  provider,  criticism  of  specific  aspects  of  health  care  delivery 
may  increase.   [112]     Perceived  access  to  care  is  an  important  component  of 
satisfaction.   [113]     Satisfaction  is  also  related  to  a  person's  general  atti- 
tude. [114] 

A  few  studies  go  beyond  the  relationship  of  satisfaction  to  patient  charac- 
teristics and  structural  factors  and  address  the  process  and  outcome  of  care, 
as  perceived  by  the  patient.     Attempts  to  relate  satisfaction  to  communication 
between  patient  and  physician,  which  measured  communication  in  terms  of  infor- 
mation items  exchanged,  have  produced  equivocal  results.   [115]     Other  investi- 
gators have  found  that  the  content  of  communication,  rather  than  necessarily 
the  amount,   influences  satisfaction.     Problems  in  communication  include  the 
tendency  of  physicians  to  seize  the  initiative  and  use  unexplained  medical 
terminology,   and  their  emphasis  on  alleviating  the  problem,   rather  than 
explaining  its  cause.     The  key  to  satisfaction  may  be  meeting  patients'  expec- 
tations.    Satisfaction  decreases  when  anticipated  behavior  (for  example, 
giving  injections)  does  not  occur  and  when  the  physician  is  expected  to  be 
friendly  and  concerned  and  is  not.     When  communication  addresses  the  patient's 
anxieties,  concerns,  and  expectations,   satisfaction  increases.    [116]  Failure 
to  provide  information  on  child  care  and  to  meet  the  patient's  educational 
needs  has  resulted  in  decreased  satisfaction.  [117] 

Patient  compliance  is  related  to  satisfaction,  but  the  relationship  is  not 
clear-cut.    [118]     Compliance  decreases  when  the  physician  is  regarded  as 
unfriendly  and  not  understanding  and  when  a  complicated  therapeutic  regimen 
is  prescribed.     Compliance  is  also  less  for  those  whose  expectations  are  not 
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met.     Compliance  has  been  shown  to  increase  when  the  patient's  condition  is 
prolonged  or  increases  in  severity,   even  though  initial  failure  to  comply 
may  have  produced  the  worsened  condition.  [119] 

Satisfaction  with  the  process  of  care  apparently  is  related  to  the  extent  to 
which  patient  expectations  are  met  and  this,   in  turn,   influences  compliance, 
but  the  relationships  need  further  exploration.     If  patient  expectations  are 
unrealistic  or  would  lead  to  inappropriate  care,  communication  between  the 
patient  and  physician  should  make  this  apparent.     With  patient  education, 
more  realistic  expectations  should  result  without  decreasing  either  satisfac- 
tion or  compliance. 

A  few  studies  have  compared  patients'  assessments  of  the  results  of  medical 
treatment  with  physicians'  estimates  of  expected  outcome.    [120]  Patients 
have  been  questioned  on  several  measures,   including  amount  of  sickness  and 
dysfunction,  the  quality  of  life  following  gall  bladder  surgery,  satisfaction 
with  postoperative  length  of  stay,   and  more  general  measures  of  outcome.  Each 
study  has  revealed  some  compatibility  between  physician  and  patient  assess- 
ments.    When  patients  report  a  less  desirable  status  than  expected,   it  fre- 
quently stems  from  social  or  psychological  limitations,  which  may  not  have 
been  considered  in  the  physician  assessment.     In  one  study,   patient  assess- 
ment was  used  as  an  initial  screen  to  detect  cases  for  which  a  more  detailed 
review  by  physicians  was  warranted.     The  validity  of  patients'   judgments  was 
shown  to  a  limited  extent  when  a  chart  review  showed  that  in  some  cases  for 
which  the  patients'  assessments  were  less  than  optimal,  there  were  correctable 
errors  in  the  process  of  care.  [121] 

The  above  work  suggests  that  the  patient  perspective  adds  an  important  dimen- 
sion to  quality  which  may  influence  compliance  and  outcome  and  assist  in  as- 
sessing quality  of  care,  even  though  that  perspective  may  not  be  adequately 
represented  in  existing  quality  assurance  programs.     The  effectiveness  of  pa- 
tient involvement  may  be  enhanced  by  adequate  health  education. 

Educating  individuals  to  alter  ingrained  behavior  patterns   in  a  manner  which 
might  improve  their  health  status  is  equally  difficult  as  convincing  physi- 
cians to  change  the  manner  in  which  they  practice  medicine.     The  inhibiting 
and  facilitating  factors  in  both  cases  are  not  adequately  understood.  Despite 
many  examples  in  which  no  change  occurred,  however,   there  are  some  successful 
health  education  programs  which  might  be  examined  and  replicated. 

Health  education  of  the  general  public  has  been  attempted  through  the  media. 
A  survey  of  viewers  of  the  television  program  "Feeling  Good"  showed  some 
impact  on  behavior,  but  conclusive  findings  await  the  results  of  a  controlled 
evaluation  in  four  cities.    [122]     The  Stanford  Heart  Disease  Prevention 
Program  covers  three  years  and  uses  a  variety  of  media  to  reach  the  target 
population  in  two  communities  and  a  control  community.    [123]  Preliminary 
findings  show  the  greatest  changes  for  the  group  which  received  intensive 
group  instructions  as  well  as  a  media  campaign.     Although  improvements 
appeared  in  parts  of  the  physical  examination,  reports  of  smoking  and  eating 
habits,   and  diet  knowledge,  no  significant  changes  occurred  in  weight,  sugar 
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intake,   serum  cholesterol,   or  leisure  activity — factors  which  may  be  more 
deeply  linked  to  one's  lifestyle. 

A  study  of  an  inner-city  population  has  shown  that  personal  contact  is  more 
effective  than  media  programs.    [124]     A  health  education  program  incorporated 
into  an  elementary  school  science  curriculum  had  a  significant  impact  and 
led  to  speculation  that  children  might  educate  their  parents  for  more  healthy 
lifestyles.   [125]     Mothers  have  been  trained  successfully  to  recognize 
indications  of  possible  strep  infection  and  to  take  throat  cultures  from 
their  children.    [126]     But  other  programs  to  educate  parents  to  care  for  their 
children  have  been  less  successful.  [127] 

Similarly  mixed  findings  result  from  educational  programs  for  patients  with 
particular  diseases  or  conditions.     A  multidisciplinary ,   individualized  pro- 
gram for  patients  with  congestive  heart  disease  syndrome  and  their  families 
resulted  in  increased  patient  knowledge  and  compliance,   improved  social  inter- 
actions,  and  reductions  in  hospital  admissions  and  length  of  stay.    [128]  A 
self-care  program  for  patients  with  asthma  led  to  a  significant  reduction  in 
emergency  room  visits  for  symptomatic  relief  and  was  estimated  to  have  a 
cost-to-benefit  ratio  of  at  least  one-to-five.    [129]  However,   less  positive 
findings  resulted  from  a  program  for  elderly  males  with  bronchitis  [130] 
and  another  intended  to  increase  compliance  with  an  antibiotic  regimen.  [131] 

Despite  the  potential  for  attitudinal  factors  and  knowledge  to  influence 
individual  behavior  and  health  status,  many  questions  remain  unanswered.  At 
the  societal  level,  however,  greater  precedent  exists  for  such  involvement. 

The  Comprehensive  Health  Planning  Act,   the  Health  Maintenance  Organization 
Act  of  1973,   the  National  Health  Planning  and  Resources  Development  Act  of 
1974,   and  the  National  Health  Education  and  Promotion  Act  of  1975,   all  in- 
clude provisions  for  consumer  involvement.     Consumer's  reactions  to  their 
health  care  is  an  expressed  concern  of  the  FY  1977-81  Forward  Plan  for  Health, 
issued  by  the  Department  of  Health,  Education,   and  Welfare.     Even  the  legis- 
lation for  Professional  Standards  Review  Organizations  includes  a  limited 
reference  to  consumers — in  states  with  three  or  more  PSROs,  which  are  there- 
fore required  to  have  a  statewide  Professional  Standards  Review  Council,  four 
persons  on  that  council  must  be  representatives  of  the  public.     A  few  states 
have  increased  non-professional  representation  on  state  licensing  boards; 
locally  the  inclusion  of  consumers  on  hospital  boards  of  trustees  is  increas- 
ing. 

The  effect  of  consumer  representation  is  very  difficult  to  evaluate — in  part 
because  of  methodological  problems  and  in  part  because  it  is  value-laden. 
The  complexities  of  the  issues  and  a  few  evaluative  attempts  have  been  de- 
scribed in  the  literature,  however.    [132]     The  rationale  behind  public  repre- 
sentation usually  assumes  that  social  services  should  be  provided  in  a  manner 
which  is  responsive  to  the  needs  and  desires  of  the  users.     Thus,  repre- 
sentatives of  users  should  be  included  in  policymaking  bodies.     As  resource 
constraints  increase,   it  is  important  that  users  appreciate  the  trade-offs 
involved  in  considering  competing  demands  and  allocating  resources.     In  regu- 
latory or  quality  control  agencies,   the  users  have  an  additional  interest  in 
assuring  that  their  health  and  safety  are  being  protected.     Ultimately,  a 
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recommendation  to  include  the  public  in  such  decisions  must  rest  on  the  judg- 
ment that  it  is  desirable.     So  it  is  with  health  care  quality  assurance  pro- 
grams . 

The  steering  committee  is  of  the  opinion  that  consumer  involvement  in  quality 
assurance  should  be  encouraged  and  expanded.   If  the  public  (both  as  individual 
patients  and  representatives  in  policymaking  bodies)  can  gain  a  better  under- 
standing of  the  determinants  of  health,  the  limitations  of  health  care,  the 
resources  required  to  provide  it,  and  the  necessity  to  work  in  partnership 
with  professionals  to  create  a  system  of  health  care,  this  should  result  in 
improvements  in  the  quality  and  appropriateness  of  health  services  and  a 
healthier  public.     Although  the  objective  is  clear,  the  methods  for  achieving 
it  are  not.     Therefore,  the  recommendations  emphasize  the  need  for  additional 
research,  demonstration,  and  evaluation. 

STEERING  COMMITTEE  RECOMMENDATIONS 

1 .  Increased  consumer  involvement  in  quality  assurance  should  begin  with 
representation  of  the  public  on  the  National  Professional  Standards  Review 
Council.     This  may  require  a  legislative  amendment. 

2.  Additional  research  is  needed  to  identify  dimensions  of  health  care  that 
are  important  from  the  consumer's  perspective,  which  can  be  incorporated  into 
valid  and  reliable  instruments  for  assessing  patient  expectations  and  satis- 
faction.    Once  the  measures  are  adequate,  the  feasibility  of  implementing  them 
in  formal  quality  assurance  programs  can  be  better  tested. 

3.  Experimental  projects  involving  consumers  in  formal  quality  assurance 
programs  should  be  funded.     Consumer  or  patient  boards  to  hear  patient  com- 
plaints, evaluate  their  validity  and  causes,  and  link  into  quality  assurance 
programs  might  be  instituted.     The  use  of  patient  questionnaires  in  assessing 
the  quality  of  care  should  be  tested.     An  assessment  should  be  conducted  of 
the  effect  of  explicitly  noting  patient  expectations  in  the  medical  record, 
providing  education  to  modify  unrealistic  expectations,  and  using  that  infor- 
mation to  guide  and  monitor  the  provision  of  patient  care.     A  more  direct  in- 
volvement of  consumers  with  providers  in  assessing  the  quality  of  care  should 
be  evaluated,  since  both  groups  might  learn  from  one  another  and  heighten 
their  appreciation  of  the  complexities  of  quality  assessment,  particularly 

as  it  relates  to  the  expenditure  of  resources. 

4.  The  provision  of  patient  education  is  an  important  consideration  in  assess 
ing  the  quality  of  care  in  instances  where  education  is  known  to  be  beneficial 
To  the  extent  that  process-oriented  criteria  are  used  to  monitor  care,  efforts 
to  include  educational  components  (such  as  dietary  instruction  for  diabetics) 
should  be  encouraged.     If  changes  in  the  process  of  care  or  delivery  settings 
are  contemplated,   the  acceptance  of  such  changes  will  be  increased  if  infor- 
mation is  provided  to  the  patients  in  advance. 
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5.  Additional  research  is  needed  to  identify  factors  associated  with  effec- 
tive health  education.     Attention  should  be  given  to  the  effect  of  alterna- 
tive    media,  differing  levels  of  patient  and  family  involvement,   the  duration 
of  behavior  change  and  whether  reinforcement  is  needed,   the  potential  contri- 
bution of  motivational  research,   and  patient  factors  which  may  influence 
effectiveness,   such  as  emotional  state,  demographic  characteristics,  and 
health  status.     Different  approaches  may  be  required  for  different  patient 
conditions,   ranging  from  preventive  care,   to  care  for  acute  illness  and 
chronic  conditions. 

6.  Because  the  treatment  process  by  definition  involves  a  health  care  pro- 
vider, additional  research  is  needed  to  explore  the  psycho-social  perspective 
of  the  provider  and,   in  particular,  to  identify  factors  associated  with  pro- 
vider satisfaction  or  stress.     The  influence  of  patients'  mannerisms  and 
conduct  on  physician  behavior  should  be  further  explored.     Greater  understand- 
ing of  these  factors  may  improve  the  patient-provider  relationship  and  health 
outcomes . 

7.  Changes  in  the  content  of  the  medical  curriculum  and  inclusion  of  material 
about  the  relationship  between  patient  and  provider  should  be  evaluated. 
Alternative  models  of  the  patient-physician  relationship  should  be  tested  to 
determine  whether  certain  models  are  more  or  less  appropriate  for  particular 
patients  or  conditions. 

8.  Existing  legislative  requirements  which  might   further  health  education 
should  be  exploited.     For  example,   informed  consent  reyuirements  might  provide 
a  unique  chance  to  educate  the  patient  about  his  condition,   rather  than  simply 
obtaining  an  unthinking  agreement  to  treatment. 

9.  The  consumer's  role  in  governance  and  policymaking  needs  careful  documen- 
tation and  analysis  so  that  more  responsible,  comfortable,   and  effective 
relationships  may  evolve. 
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Appendix  A 


RESEARCH  AGENDA 


Suggested  research  topics  are  presented  in  hopes  of  stimulating  the  develop- 
ment of  a  more  comprehensive  research  strategy  for  health  care  quality  assurance. 
Research  findings  should  enable  policymakers  to  understand  the  potential  gains 
in  health  status  and  reductions  in  expenditures  that  result  from  quality  assur- 
ance and  rationally  allocate  funds  for  the  development  and  operation  of  health 
care  quality  assurance  programs. 

Research  needs  fall  into  three  general  categories:     research  to  develop 
more  reliable  and  valid  assessment  tools  to  measure  the  level  of  quality; 
research  to  foster  the  improvement  or  assurance  of  quality,  which  involves  the 
development  of  better  methods  for  altering  the  behavior  of  both  health  care  pro- 
viders and  consumers;   and  research  to  develop  better  evaluation  measures  to 
determine  the  extent  to  which  resources  devoted  to  quality  assurance  activities 
have  succeeded  in  improving  the  health  status  of  the  population. 

Research  topics  are  categorized  according  to  the  major  divisions  of  the 
summary  in  Chapter  1. 

GENERAL  RESERARCH  AND  EVALUATION  NEEDS 

A  systematic  evaluation  of  the  effectiveness  of  both  federal  and  pri- 
vately sponsored  health  care  quality  assurance  programs  should  be  designed 
and  implemented.     The  research  strategy  should  include  geographic  areas  with 
and  without  quality  review  programs,   as  well  as  areas  of  otherwise  similar 
characteristics  but  different  types  of  review.     Specific  research  questions  and 
measurement  needs  are  presented  throughout  this  appendix. 

Improved  methods  are  needed  to  identify  and  aggregate  the  effects  on 
health  status  that  result  from  health  care.     Summary  measures  must  be  developed 
to  relate  improvements  in  health  status  to  the  activities  or  resources  involved 
in  producing  them,   and  thus  permit  an  assessment  of  the  additional  benefit  from 
continuing  medical  education,   quality  assurance,   and  other  activities  designed 
to  improve  the  quality  of  care. 

The  systematic  accumulation  of  data  to  describe  current  patterns  of  medical 
care  in  all  settings  (hospital,   long-term,  and  ambulatory)  is  an  important 
research  priority.     Special  attention  should  be  given  to  unusual  methods  of 
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treatment,   the  extremes  of  under-  and  over-utilization,   and  the  reasons  for 
such  variation.     This  information  should  be  used  to  estimate  current  levels 
of  inappropriate  care  and  the  margin  by  which  care  might  reasonably  be  improved. 
It  would  then  be  possible  to  determine  the  required  magnitude  of  quality  assur- 
ance activites,   identify  particular  patients,  providers,  or  conditions  in  need  of 
special  attention,  and  design  quality  assurance  mechanisms  to  meet  those  needs. 

Research  is  needed  to  determine  the  extent  to  which  factors  determined  by 
public  policy,  but  outside  the  process  of  care,  influence  the  quality  of  care  and 
health  status.     Such  factors  include  the  availability  and  accessibility  of  care, 
links  between  levels  of  care,  the  comprehensiveness  of  benefits  and  reimbursement 
policies  of  insurance  carriers,  and  the  organizational  arrangements  in  which 
health  care  is  provided. 

Better  methods  are  needed  to  assess  unmet  health  needs  and  under-utilization 
of  services  so  that  quality  assurance  programs  can  extend  their  responsibilities 
to  deal  with  these  problems. 

Methods  are  needed  to  integrate  information  on  care  provided  in  all  settings 
so  that  quality  assessment  can  be  based  on  all  services  provided  to  a  person  over 
t  ime . 

Alternative  data  sources  are  needed  to  expand  the  information  captured  by 
the  minimum  data  sets  for  hospital,  ambulatory,  and  long-term  care  and  avoid 
undue  reliance  on  the  medical  record.     The  costs  of  routinely  retrieving  such 
data  should  be  determined,  as  well  as  the  benefits  which  accrue  from  their 
availability. 

Experimental  curricula  should  be  developed  and  tested  for  courses  in  health 
care  evaluation  for  all  health  care  professionals. 

Better  techniques  are  needed  to  determine  the  impact  of  quality  assurance 
programs  on  utilization  and  cost  of  medical  care.     In  this  regard,   several  spe- 
cific studies  are  needed,  as  listed  below. 

Factors  which  affect  changes  in  admission  rates  and  length  of  stay  should 
be  analyzed,   including  existing  length  of  stay,  occupancy  rate,  demand  for  beds 
in  the  facility  and  elsewhere  in  the  community,  provider  and  consumer  character- 
istics, case  mix,  and  the  relationship  of  revenues  to  costs.     The  interrelation- 
ships among  these  factors  may  suggest  areas  for  concentration  in  quality  or 
utilization  review. 

Factors  affecting  the  effectiveness  of  quality  assurance  systems  should  be 
delineated  and  their  relative  contributions  assessed.     Factors  may  include: 
methods  of  review,  corrective  action  programs,  existing  levels  of  quality,  the 
influence  of  discharge  planning,  availability  of  appropriate  alternative  care, 
peer  and  community  pressure,   and  the  role  of  planning,  rate  setting,   and  moni- 
toring agencies. 
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Better  methods  must  be  developed  to  estimate  total  costs  and  unit  costs 

for  quality  assurance  activities  by  function  and  activity,   including  costs 

of  services  obtained  gratis  or  below  true  costs.     Better  methods  are  needed 

to  allocate  joint  costs  and  to  measure  work  units  required  by  management  for 

purposes  in  addition  to  quality  assurance. 

The  relative  cost  and  effectiveness  of  various  quality  assessment  tech- 
niques applied  to  different  provider  and  consumer  groups  should  be  assessed, 
including:     prior  authorization  and  second  opinions;   targeted  reviews  deter- 
mined on  the  basis  of  provider,  patient,   and  diagnosis;  and  corrective 
action  programs  (singly  or  in  combination)  for  both  consumer  and  provider. 

Better  methods  are  needed  to  estimate  societal  savings  from  quality 
reviews,  based  on  capital  expenditures  avoided  and  other  deterrent  effects. 

Better  methods  are  needed  to  estimate  bed  days  saved  from  both  admissions 
denied  and  length  of  stay  reduced,  which  take  into  account: 

•  cost  of  alternative  care  required  as  a  result  of  either  admission 
denials  or  reductions  of  length  of  stay; 

•  short-term  and  long-term  changes  in  per  diem  associated  with: 
changes  in  occupancy  rates,  by  initial  rate  and  size  of  change; 
level  of  per  diem;  proportion  of  variable  costs  to  total  costs; 
type  of  facility  and  organization;  amount  of  revenues  in  excess 
of  costs;  population  changes  within  a  geographic  area;  and 
characteristics  of  outside  authorities,  such  as  planning  and 
rate-setting  agencies; 

•  short-term  and  long-term  community  effects  from  interhospital  shifts, 
based  on  case  studies  and  simulations;  and 

•  capital  expenditures  avoided. 

The  assumptions  regarding  incentives  for  hospitals  to  conduct  meaningful 
utilization  review  programs  should  be  examined  to  determine  whether  new  incen- 
tive arrangements  should  be  developed  and  tested. 

Analyses  of  the  range  of  costs  for  hospital  PSRO  activities  should  be 
conducted  and  related  to  the  effectiveness  of  review.     The  manner  in  which 
costs  are  reported  before  and  after  hospitals  affiliate  with  PSROs  deserves 
special  attention. 

The  cost  and  cost-effectiveness  of  different  arrangements  for  delegating 
PSRO  review  requirements  to  hospitals  should  be  determined,   including  fully 
delegated  hospitals,   partially  delegated  hospitals,   and  non-delegated  hospitals, 
taking  into  consideration  both  the  type  of  review  which  is  delegated  (concurrent, 
medical  care  evaluations,  or  profile  analysis)  and  the  review  personnel  which 
are  delegated  (physician  advisors  or  review  coordinators). 


130 


Criteria  should  be  developed  for  categorizing  successful  and  unsuccessful 
MCEs  and  isolating  factors  associated  with  success,   so  that  more  effective  MCEs 
may  evolve.     Data  bases  must  be  developed  to  describe  current  MCEs,   so  that  a 
more  definitive,   future  assessment  of  effectiveness  can  be  made. 

A  wide  range  of  innovation  and  evaluation  should  be  encouraged.     The  effec- 
tiveness of  concurrent  and  prospective  MCEs  that  permit  direct  intervention  in 
the  process  of  care  where  warranted  should  be  tested.     The  relative  merits  of 
areawide  MCEs,  as  opposed  to  individual  hospital-based  MCEs,  should  be  assessed. 
This  will  require  the  resolution  of  an  apparent  dilemma.     The  success  of  MCEs 
has  been  said  to  rest  on  medical  staff  involvement  in  selecting  topics,  estab- 
lishing criteria,  and  reviewing  the  results.     Yet,  without  the  involvement  of  an 
external  monitor,  such  efforts  could  be  self-serving.     Experimentation  with 
a  mix  of  internally  and  externally  initiated  MCEs  may  result  in  an  optimal 
balance . 


ASSESSMENT  OF  HEALTH  OUTCOMES 

Better  measures  for  assessing  health  outcomes  in  terms  of  both  health 
status  and  patient  satisfaction  should  be  developed.     Proxy  (substitute)  outcome 
measures  are  needed,  such  as  immunization  levels,  as  well  as  proximate  or  inter- 
mediate outcomes,  which  occur  closer  in  time  to  the  provision  of  care  than  end- 
result  outcome  measures. 

The  relationship  between  process,   intermediate  outcome,  and  final  outcome 
measures  should  be  established. 

Special  studies  are  needed  to  develop  and  evaluate  simplified  methods  for 
including  outcome  assessments  in  the  routine  PSRO  review  requirements. 

Additional  research  to  establish  the  natural  history  of  diseases  and  the 
efficacy  of  medical  procedures  and  therapies  should  be  funded.     For  research 
findings  to  be  useful  in  quality  assessment,  determinations  of  efficacy  should 
be  made  under  average,  as  well  as  ideal,  treatment  situations  at  various  points 
in  time  and  should  include  a  broad  range  of  outcome  measures.     As  an  initial 
step,  a  review  of  current  knowledge  of  the  efficacy  of  common  medical  procedures 
should  be  conducted  to  assist  in  establishing  research  priorities. 

The  multiple  factors  that  influence  patient  outcome  and  health  status 
should  be  better  specified.     To  accomplish  this,  undesirable  outcomes  could 
be  identified  in  a  large  population  and  traced  backwards  through  the  process 
of  care,  using  record  reviews,   interviews  of  patients  and  providers,  and  other 
techniques,   to  isolate  reasons  for  poor  outcome.     Reasons  attributable  to  the 
medical  care  process  should  be  considered  in  designing  medical  care  and  quality 
assurance  programs. 

Experimental  projects  in  which  patients  and  physicians  jointly  establish 
outcome  objectives  for  patient  care  should  be  conducted  in  order  to  determine 
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the  extent  to  which  patient  involvement  influences  patient  satisfaction  and 
health . 

AMBULATORY  CARE 

Probability  sampling  techniques  must  be  developed  for  ambulatory  quality 
assessment  based  on  claims  review,   so  that  review  can  focus  on  patients  and 
providers  who  fall  at  the  extremes  of  distributions  of  care  patterns  and  at 
the  same  time  give  estimates  of  the  broader  spectrum  of  care  provided  to  the 
total  population. 

In  a  computerized  ambulatory  claims  review  system,   the  costs  and  effec- 
tiveness of  different  screens  and  rejection  rates  should  be  analyzed  using: 
only  physician  claims;  physician  and  prescription  claims;  physician,  pre- 
scription,  and  laboratory  claims;   all  ambulatory  claims;  non-physician 
reviewers;  physician  reviewers;  and  both  non-physician  and  physician  reviewers. 

Methods  for  supplementing  information  on  the  ambulatory  claim  form  should 
be  developed  and  tested,  using  the  Minimum  Ambulatory  Care  Data  Set  as  a  base. 
One  project  should  test  the  value  of  diagnostic,   patient,   and  laboratory  regis- 
tries to  facilitate  problem  identification  over  time. 

Quality  assessment  techniques  which  appropriately  address  the  unique  aspects 
of  primary  ambulatory  care  should  be  developed.     This  requires  attention  to 
classification  of  signs  and  symptoms,   rather  than  diagnoses.     Clinical  skills 
should  be  assessed,   including  elicitation  of  signs  and  symptoms  and  their  history, 
performance  of  a  physical  exam,   the  synthesis  of  information  into  recommendations 
for  care,  and  determination  of  the  appropriate  point  for  referral.   Techniques  for 
assessing  the  coordination  of  care  for  a  single  individual  over  time  should  also 
be  developed,   since  the  coordination  function  is  one  of  the  key  components  of 
primary  care. 

LONG-TERM  CARE 

Demonstration  projects  to  create  community-based  long-term  care  quality 
assurance  programs,   as  described  in  the  body  of  this  report,   should  be  funded 
and  evaluated.     They  should  include  representatives  of  all  groups  and  facili- 
ties involved  in  providing  long-term  care.     Review  should  emphasize  functional 
assessments . 

The  proposed  Minimum  Data  Set  for  Long-Term  Care  should  be  tested  and 
evaluated.     Developmental  work  is  needed  to  further  refine  the  definitions  for 
mental  functioning,   social  functioning,  certain  aspects  of  physical  function- 
ing,  and  the  events  or  reasons  for  use  of  services  other  than  diagnoses. 

Experimental  reimbursement  demonstrations  should  be  funded  to  test 
reimbursement  policies  that  provide  incentives  to  improve  the  appropriateness 
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and  quality  of  long-term  care.     Reimbursement  based  on  capitation  might  enable 
the  individual  patient  to  move  from  one  level  of  care  to  another  without 
being  penalized.     Alternatively,   facility  reimbursement  rates  based  on  the 
usual  mix  of  patients,   rather  than  specific  patients,  might  permit  patients 
to  be  moved  from  one  level  of  care  to  another,  depending  on  their  conditions. 

Longitudinal  research  is  needed  to  determine  the  effect  of  the  availability 
of  different  long-term  care  supportive  services  on  patient  outcome.  Comparable 
groups  of  persons  receiving  different  combinations  of  long-term  care  services 
should  be  evaluated  to  determine  whether  outcomes  achieved  are  similar  and  at 
what  relative  costs.     The  influence  of  state  Medicaid  regulations  and  benefits 
on  quality  of  long-term  care  should  be  assessed. 

More  basic  research  is  needed  to  develop  more  appropriate  techniques  for 
quality  assessment  that  consider  both  the  health  service  and  quality  of  living 
aspects  of  long-term  care. 


IMPLICATIONS  FOR  PROVIDER  PERFORMANCE 

Extensive  research  is  needed  to  devise  methods  for  encouraging  improvements 
in  patient  care,  once  deficiencies  are  identified.  Relevant  literature  from  the 
social  sciences,  as  well  as  from  medical  education,  should  be  utilized.  Existin 
quality  deficiencies  should  be  categorized  to  assist  in  determining  the  reasons 
for  their  occurrence  and  the  design  of  appropriate  corrective  actions.  All  meth 
ods  for  improvement  should  be  carefully  evaluated  to  determine  the  extent  to 
which  they  result  in  lasting  behavior  change. 

Factors  associated  with  effective  continuing  medical  education  should  be 
identified,   so  that  more  successful  programs  can  be  developed. 

Hypotheses  about  optimal  learning  conditions  in  other  fields  should  be  fur- 
ther explored  in  medical  settings.     Research  on  the  diffusion  of  medical  infor- 
mation may  identify  potential  points  of  intervention  to  hasten  the  process  of 
information  exchange. 

Different  methods  for  informing  physicians  about  instances  in  which  their 
care  differs  from  local  norms  should  be  evaluated,   including  simple  unstruc- 
tured feedback  of  information,  a  more  targeted  response  concentrating  on  par- 
ticular areas,  or  the  provision  of  incentives  for  review  and  change.  The 
reasons  for  failure  to  change  should  be  explored. 

The  influence  of  the  organization  of  health  care  resources  on  quality  needs 
detailed  analysis.     The  impact  of  working  arrangements  which  make  performance 
more  visible  and  facilitate  the  sharing  of  information  should  be  evaluated. 
The  potential  for  informal  learning  in  medical  practice  should  be  examined. 

For  instances  in  which  clearly  inappropriate  care  is  identified  and 
behavior  does  not  change,  sanctions  which  are  less  drastic  than  permanent  loss 
of  licensure  may  be  more  readily  applied.     Experimentation  with  intermediate 
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sanctions  should  be  conducted,   including  temporary  curtailment1  of  privileges, 
licensing  with  restrictions  on  specified  areas  of  practice,  mandatory  super- 
vision of  medical  practice,   remedial  education,   or  curtailment  of  privileges 
for  certain  procedures  (perhaps  surgical)  which  are  seldom  performed. 

Demonstrations  should  test  the  effectiveness  of  equipping  PSROs  with  a 
wider  range  of  sanctions  for  clearly  inappropriate  behavior,   including  more 
direct  links  with  licensing  bodies  or  perhaps  authorizing  the  PSRO  to  remove 
a  license  with  due  cause. 

The  feasibility  and  effectiveness  of  publicizing  instances  of  persistently 
poor  quality  by  individual  practitioners  in  public  media  should  be  explored. 

CONSUMER  INVOLVEMENT  IN  QUALITY  ASSURANCE 

Additional  work  is  needed  to  identify  dimensions  of  health  care  that  are 
important  from  the  consumers'   perspective,  which  can  then  be  incorporated  into 
valid  and  reliable  instruments  for  assessing  patient  expectations  and  satis- 
faction.    Similarly,  additional  work  is  needed  to  relate  expectations  and 
satisfaction  to  compliance  with  medical  instructions  and  to  health  outcome. 

Experimental  quality  assurance  programs  should  include  consumer  or  patient 
boards  to  hear  patient  complaints  and  evaluate  their  validity  and  causes.  The 
use  of  patient  questionnaires  in  assessing  quality  should  be  explored.  Patient 
expectations  upon  seeking  care  might  be  determined  and  used  as  the  basis  for 
providing  patient  education  and  instituting  treatment;   the  influence  of  expec- 
tations on  compliance  and  outcome  could  then  be  determined.     A  more  direct 
involvement  of  consumers  with  providers   in  assessing  the  quality  of  care  should 
be  tested. 

Additional  research  is  needed  to  identify  factors  associated  with  effec- 
tive health  education.     Attention  should  be  given  to  the  effect  of  alternative 
media,  differing  levels  of  patient  and  family  involvement,   the  duration  of 
behavior  change  and  whether  reinforcement  is  needed,   the  potential  contribution 
of  motivational  research,  and  patient  factors  which  may  influence  effectiveness, 
such  as  emotional  state,  demographic  characteristics,  and  health  status.  Differ- 
ent approaches  may  be  required  for  different  patient  conditions,   ranging  from 
preventive  care  to  acute  illnesses  and  chronic  conditions. 

In  elementary  and  secondary  schools,   experimental  health  education  programs 
should  be  incorporated  in  the  basic  curricula.     The  influence  on  the  health 
behavior  of  students  and  adults  with  whom  they  have  contact  should  be  evaluated. 

Analyses  are  needed  of  the  decision-making  processes  by  which  a  consumer 
elects  to  adopt  a  healthful  lifestyle,   seek  appropriate  medical  care,  or 
comply  with  medical  instructions. 
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The  manner  in  which  the  American  lifestyle  may  contribute  to  national 
health  problems  should  be  examined  so  that  the  feasibility  of  encouraging 
healthful  habits  in  a  society  that  simultaneously  encourages  unhealthful  habits 
might  be  better  analyzed. 

The  experiential  factors  in  being  sick  should  be  examined  so  that  fac- 
tors which  induce  unnecessary  stress  might  be  reduced  in  quality  medical  care. 

Because  the  treatment  process  by  definition  involves  a  health  care  pro- 
vider, additional  research  is  needed  to  explore  the  psycho-social  perspective 
of  the  provider  and,   in  particular,  to  identify  factors  associated  with  pro- 
vider satisfaction  or  stress.     The  influence  of  patients'  mannerisms  and  con- 
duct on  physician  behavior  should  be  further  explored.     Greater  understanding 
of  these  factors  may  improve  the  patient-provider  relationship  and  health 
outcomes . 

Changes  in  the  content  of  the  medical  curriculum  and  inclusion  of  material 
about  the  relationship  between  patient  and  provider  should  be  evaluated. 
Alternative  models  of  the  patient-physician  relationship  should  be  tested  to 
determine  whether  certain  models  are  more  or  less  appropriate  for  particular 
patients  or  conditions. 

Opportunities  in  existing  legislative  requirements  to  further  health 
education  should  be  exploited.     On  an  experimental  basis,   the  requirement  for 
informed  consent  may  provide  a  unique  chance  to  educate  the  patient  regarding 
many  aspects  of  his  condition,  rather  than  simply  obtaining  an  unthinking 
agreement  to  treatment. 

The  consumer's  role  in  governance  and  policy-making  needs  careful  docu- 
mentation and  analysis,  so  that  more  responsible,  comfortable,  and  effective 
relationships  with  health  care  professionals  may  evolve. 
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APPENDIX  B 


QUALITY  ASSURANCE  PROGRAMS  VISITED 


Bethesda  Lutheran  Medical  Center 
St.  Paul,  Minnesota 

Foundation  for  Health  Care  Evaluation 
Minneapolis,  Minnesota 

Colorado  Foundation  for  Medical  Care 
Denver,  Colorado 

Utah  Professional  Standards  Review 

Organization 
Salt  Lake  City,  Utah 

Multnomah  Foundation  for  Medical 
Care 

Portland,  Oregon 

Medical  Care  Foundation  of 

Sacramento 
Sacramento,  California 

Foundation  for  Medical  Care  of 

San  Joaquin 
Stockton,  California 

San  Joaquin  Area  Professional 

Standards  Review  Organization 
Stockton,  California 


The  Permanente  Medical  Group 
Oakland,  California 

Kaiser  Foundation  Health  Plan,  Inc. 
Los  Angeles,  California 

University  of  California,  Los 

Angeles  -  EMCRO 
Los  Angeles,  California 

Indian  Health  Service 
Tucson,  Arizona 

State  Department  of  Public  Welfare 
Austin,  Texas 

Overlook  Hospital 
Summit,  New  Jersey 

The  Mount  Sinai  Hospital 
New  York,  New  York 

Columbia  Medical  Plan 
Columbia,  Maryland 

National  Capital  Medical 

Foundation 
Washington,  D.  C. 
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APPENDIX  C 


DESCRIPTORS  OF  QUALITY  REVIEW  SYSTEMS 

DESCRIPTION  OF  QUALITY  REVIEW  SYSTEM  (to  be  obtained  through  written  material 
and  telephone  conversations  in  advance  of  potential  site  visit): 

I.     Origins  and  Summary  of  Review  System 

A.  General  objectives  and  goals 

B.  Groups,   events  and  motivations  stimulating  interest  in  quality 
assurance  and  reasons  for  selecting  the  chosen  approach  to  review 
(Note  involvement  of  health  professionals,   administrators,  third 
parties,   and  consumers.) 

C.  Dates 

1.  Date  developmental  activities  began 

2.  Date  on  which  review  activities  were  ongoing  on  a  routine 
basis 

II.     Structure  of  the  System 

A.  Sponsorship,   administrative  unit,  and  financial  support 
(developmental  and  operational) 

B.  Inter-relationships  between  the  review  system  and  involved  groups 
or  parties  external  to  the  system  (Note  specific  involvement  of 
consumers,  public  agencies,  or  other  persons  external  to  the  program 
under  review. 

C.  Scope  of  system 

1.     Population  variables 

-  Geographical  or  institutional  base 

-  Type  of  population  included 

-  Extent  to  which  review  focuses  on  a  population  of  potential 
users;  questions  of  access,  non-utilization,  and  under- 
utilization;  and  a  system  of  health  care  beyond  the  immediate 
episode  of  illness 
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2.     Provider  variables 

-  Single  vs.  multiple  institutions 

-  Numbers  and  types  of  providers  included 
D.     Organization  chart 

III .     Review  Process 

A.  Types  of  health  care  problems  included 

B.  Unit  of  analysis 

C.  Types  of  review 

D.  Criteria  for  assessment 

E.  Data  requirements  (including  confidentiality  measures  and  require- 
ments) 

F.  Corrective  action 

1.  Mechanisms 

2.  Process  for  notification  and  appeals 

3.  Monitoring  methods 

G.  General  description  of  review  process 

1.  Personnel  involved  and  their  responsibilities,   including  specific 
reference  to  involvement  of  patient/consumers 

2.  Summary  of  the  process  and  flow  of  information 

H.  Cost  of  conducting  review 
IV.  Results 

A.  Procedures  to  assess  attainment  of  goals  and  methods  for  revising 
goals 

B.  Evidence  of  goal  attainment 

1.  Impressions  and  anecdotes 

2.  Numbers  of  occasions  in  which  further  review  appeared  warranted 
on  the  basis  of  initial  screening 
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3.  Numbers  and  types  of  occasions  where  corrective  action  was 
instituted 

4.  Changes  in  aggregate  utilization  and  other  statistics 

5.  Other  evidence  of  impact  or  results  of  special  studies 

6.  Cost-effectiveness 

C.     Awareness  of,  and  reactions  to  the  review  system  on  the  part  of 
patient/consumers,  public  agencies,  and  physicians  outside  the 
administrative  structure  of  the  program 

V.     Long-Range  Implications 

A.  Responsiveness  to  potential  need  for  change  brought  about  through: 

1.  Increased  scientific  knowledge  and  changing  medical  practice 

2.  Social  and  economic  needs 

B.  Expectations  of  future  status  2-5  years  from  now 

C.  Underlying  problems  noted  by  the  program  (either  generic  or  specific 
to  the  setting)  which  require  further  deliberation;  examples  of  issues 
that  they  would  have  handled  differently  if  they  had  the  chance 

SUBJECTIVE  ASSESSMENT  BY  SITE  VISITORS 

I.     Current  status  and  accomplishments  to  date 

A.     Impressions  and  anecdotes 

II.     Unique  characteristics  which  may  be  influential  in  determing  the 
system's  effectiveness 

III.     Unresolved  issues  (administrative,   substantive /methodological ,  policy) 
within  system 

IV.     Required  follow-up 

V.     Unanticipated  factors  which  may  have  influenced  IOM's  ability  to  get 
information 

A.  On-site 

B.  Follow-up 
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APPENDIX  E 
Recommended  Uniform  Basic  Data  Sets* 


Ambulatory  Care  Encounter  Data  Set 


Proposed  Long-Term  Care  Data  Set 


Demographic  Data 


Name  and  unique  ID  number 
Address  and  ZIP  code 


Date  of  birth 


Sex 


Name  and/or  unique  ID  number 

Address  and  ZIP  code  or  census  tract 

a.  Latest  non-institutional 
domicile 

b.  Current  domicile  (if  different) 

Living  arrangements 

a.  Type  of  domicile 

b.  Availability  of  able  and 
willing  personal  "caregiver" 

Date  of  birth 

Sex 

Current  marital  status 
Race/ethnicity 


Individual  Attributes 


Reason  for  enounter  (principal 
problems,  complaints,  symptoms 
in  patient's  own  words) 

Principal  and  other  diagnoses 
and/or  problems  occasioning 
current  encounter  or  requiring 
treatment 

Findings 


Events/reasons  for  use  of  services 
other  than  diagnoses  (as  determined 
by  responsible  agency) 

Principal  and  other  diagnoses 
occasioning  current  use  of  services 
or  influencing  current  status 


Physical  impairments 

Physical  functioning/disability 

Mental  functioning/disability 

Social  functioning/disability 

Performance  of  independent  living 
activities 


Distress /mood/ pain/ self -percept  ion 
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Ambulatory  Care  Encounter  Data  Set 


Proposed  Long-Term  Care  Data  Set 


Service  and  Administrative  Elements 


Place  of  encounter  by  type 
(office,  clinic  or  center, 
OPD,  ER,  other) 

Provider  name,  professional 
address,  and  unique  ID  number 

Professional  category  and 
specialty  of  provider 

Encounter  date 


All  services  and  procedures 
performed  or  ordered  during 
encounter 


Disposition 

Expected  source  of  payment 


Principal  facility/ agency/provider 
ID  and /or  unique  number 


Last  principal  provider  ID 
(within  12  months) 


Itemized  charges 


Admission/entry  date  (when 
appropriate ) 

Discharge/termination  date  (when 
appropriate ) 

Assessment  date 

Category  of  services  provided  since 
last  assessment  date  or  currently 
(preventive,  acute  episodic, 
evaluative,  rehabilitative, 
support  ive ) 

Disposition  (when  appropriate) 

Sources  of  payment  (medical 
insurance,  social  services,  and 
income  maintenance) 

Costs/charges/prices  per  unit  or 
episode  of  services 


*  Jane  H.  Murnaghan,  "Review  of  the  (Long-Term  Health  Care  Data)  Conference 
Proceedings,"  Medical  Care  14  Supplement  (May  1976):  16. 
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